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1  Introduction 

1.1  Introduction  and  Objective 

In  the  last  forty  years,  robotics  has  found  widespread  use  in  many  industricd  areas.  Industrial  robots  have 
been  used  since  the  late  60’s  in  manufacturing  for  spot  and  arc  welding,  machine  loading  and  unloading, 
pcdletizing,  spray  painting  etc.  Recently  more  advanced  techniques  have  allowed  robotics  to  enter  more  skiU- 
dememding  fields  such  as  parts  assembly  [43,  6,  36],  quality  control  processes  [72],  medicine  [46,  63,  21,  4,  22] 
and  aero-space  applications  [51]. 

Some  of  the  technologic2d  advtinces  that  have  enhcinced  robotics  are  machine  vision  [62],  tactile  and  force 
sensors  [24,  12,  11],  sophisticated  control  strategies  [39,  52,  53,  59],  high-speed,  high-precision  actuators 
[1,  55,  57,  50],  dextrous  end-effectors  [20,  67,  35,  38,  12,  11],  telerobotics  [10],  etc.  As  these  technologies 
mature,  robot  manipulators  wiU  be  able  to  perform  most  of  the  labor-intensive  industrial  tasks. 

Although,  there  exist  a  liirge  number  of  avzulable  advanced  robotics  technologies,  there  are  still  many 
areas  of  the  industry  where  the  application  of  robotics  is  uncommon.  The  food  industry  is  one  such  examples 
[40].  However,  initizd  research  in  automated  food-handling  has  proved  that  robotics  has  a  potential  impact 
in  three  mmn  fields  [16]:  food  production  [33,  64,  26],  food  processing  [42,  41,  54,  19]  and  food  services 
[18]. 

Domestic  thermostabilized  food  packaging  companies,  including  the  DLA  (Defense  Logistic  Agency) 
suppliers  of  combat  ration  MRE  (Mead  Ready  to  Eat)  are  not  using  available  advanced  meinufacturing 
technology.  The  manufacturing  of  combat  rations  is  still  very  labor  intensive.  The  reliance  on  labor 
intensive  efforts  severely  impacts  the  ability  of  the  DLA  to  mobilize  domestic  resources  to  meet  emergency 
needs  in  a  cost  and  time  effective  way.  In  addition,  these  advanced  technologies  wiU  contribute  to  the 
competitive  posture  of  domestic  industries. 

The  objective  of  this  report  is  to  study  the  feasibility  of  applying  robotics  and  machine  vision  to  the 
quality  control  of  combat  rations  (in  particidar  to  the  in-line  inspection  of  MRE  pouches).  One  of  the 
goals  of  this  research  is  to  develop  simulation  eind  aneilysis  tools  that  can  help  to  evaluate  the  applicability 
of  such  advanced  technologies.  Another  gocil  is  to  present  an  initial  workable  system  that  can  be  used  as 
a  “proof- of- concept”  stating  point  for  future  developments. 

What  follows  is  a  summary  of  the  general  aspects  of  robotics  and  machine  vision  in  the  food  manufac¬ 
turing  industry.  After  this  overview  is  a  presentation  of  the  research  project  that  generated  this  work  and 
the  final  report  outline. 
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1.2  Robot  Manipulation  in  the  Food  Industry 

It  is  important  at  this  point  to  clcirify  the  difference  between  fixed-automation  and  flexible-automation 
(robotics).  Mzmy  food  industries  are  highly  automated,  using  specialized  machines  to  do  automatic  cutting, 
filling,  packaging,  palletizing,  large-sccile  movement  of  materials,  etc..  These  fixed- automation  machines 
replace  efficiently  humcin  labor  and  are,  in  general,  specifically  designed  to  perform  one  given  task. 

Flexible-automation  systems  are  also  used  to  perform  labor-intensive  tasks,  however,  those  robotics 
systems  cire  flexible,  progreimmable  and  with  small  modifications  can  easily  perform  a  VMiety  of  task 
[58,  31,  3].  This  flexibility  is  due  mairdy  to  the  presence  of  programmable  computers  in  the  control  chaiin 
and  gener^dized  mechanical  design  (robot  cirms  like  Unimate  Pmna,  IBM  Scara,  Seiko  RT-300,  etc). 

As  opposed  to  fixed-automation  systems,  computer  controlled  systems  can  be  programmed  to  make 
decisions  based  on  external  sensing  [70].  This  is  one  of  the  reasons  why  robots  equipped  with  vision 
systems  and  sensor  kernels  are  used  in  jobs  were  different  manipulation  actions  are  required  for  different 
environment  states  [9,  32,  47]. 

Fixed-automation  mechanisms  are  used  in  two  kind  of  processes,  the  ones  that  required  little  dexterity 
(labeling,  filling,  etc),  and  were  it  is  possible  to  use  a  mechanical  device  that  repeatedly  performs  the  same 
task  (sealing,  tapping,  etc).  On  the  other  hand,  the  robotics  systems  can  hEmdle  more  complex  situations. 
First,  because  the  computer  controller  permits  a  more  sophisticated  task  planing.  Second,  because  the 
mechanical  design  is  generalized  so  that  the  device  can  adapt  in  real-time. 

Large  research  efforts  have  been  spent  on  the  area  of  object  grasping.  One  of  the  problems  that  medces 
difficult  the  application  of  robotics  in  food  industry  is  the  grasping  and  handling  of  objects  with  unknown 
position/orientation.  The  solution  of  this  problem  may  require  sophisticated  detection  devices.  However 
some  investigators  have  developed  special  techniques  to  simplify  the  problem.  Go’dberg  [29,  28]  discussed  a 
Bayesian  grasping  method  in  which  basic  mainipulation  is  used  to  orient  an  object  whose  initied  orientation 
is  unknown.  The  technique  is  applicable  to  heurd  objects  with  a  polygonal  shape  (e.g.  carton  boxes  or 
plastic  containers). 

Unfortimately  not  all  objects  have  well  defined  shapes  (piece  of  ham)  mciking  it  difficult  to  apply  these 
techniques.  Nevertheless,  several  schemes  have  been  developed  to  deal  with  these  new  situations.  Lee  et 
al.  [44]  presented  a  system  using  artificicd  intelligence  and  robot  vision  to  determine  the  most  optimal  way 
to  grasp  and  manipulate  irregular  shaped  objects. 

The  stiffness  of  the  object  can  invalidate  meiny  handling  techniques.  Very  soft  food  products  present 
a  problem  for  a  steindard  robot  end-effector.  Brett  et  al.  [7]  analyzed  the  problem  of  non-rigid  product 
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handling.  Among  the  non-rigid  products  are  most  of  the  food  products.  Tedford  [64]  analyzed  the  problem 
of  end-effectors  for  soft  fruit  packing  Pneumatic  and  electro- mechamcal  grippers  were  discussed  together 
with  the  control  strategies.  A  comparison  between  human  and  machine  prehension  was  presented.  Tedford 
established  that  a  reliable  system  must  include  a  soft-touch  feedback  sensor. 

Khodabandehloo  [42]  studied  the  specific  problem  of  robotic  handling  and  packing  of  poultry  products. 
Due  to  shape  irregularities  of  poultry  products,  special  end-effectors  were  reqmred.  Some  examples  are 
contour  adapting  vacuum  grippers,  pneumatic  rubber  muscle  fingers,  moving  belt  tipped  fingers,  angle  jaw 
grippers,  etc.  In  addition  to  special  grippers  a  machine  vision  system  is  required  to  detect  position  and 
orientation  of  the  pieces  to  be  grasped. 

1.3  Machine  Vision  in  the  Food  Industry 

1.3.1  Robots  and  Vision  Systems 

From  the  examples  cited  in  the  previous  section  it  is  cleM  that  a  sophisticated  handling  device  requires 
feedback  information  from  the  external  environment.  Sophisticated  interaction  needs  knowledge  about  the 
external  conditions.  Such  knowledge  is  used  to  take  new  control  decisions  or  to  correct  errors. 

Several  sensors  can  be  used  to  capture  the  conditions  of  the  environment  (ultrasonic  proximity  sensors, 
laser  beams,  magnetic  sensors,  etc).  Machine  vision  (also  called  robot  vision)  is  one  of  the  techniques  [37] 
used  to  register  in  real-time  the  conditions  of  the  environment.  In  this  case  the  images  are  processed  to 
extract  simple  features  like  object  size,  shape,  position  md  orientation.  This  information  is  then  used  by 
the  robot  controller  to  plan  the  task  execution. 

Vision  systems  permit  the  use  of  robot  mcinipulators  in  application  that  demand  great  ability.  For 
example  Heirrel  et  al.  [33]  developed  a  prototype  robot  for  the  citrus  harvesting.  The  system  uses  robot 
vision  and  ultra  sonic  sensors  to  guide  the  special  end-effector  to  the  target. 

Khodabandehloo  [42]  uses  a  machine  vision  system  to  deal  with  cut  portions  of  poultry.  The  vision 
system  is  used  to  determine  the  orientation/position,  the  type  of  poultry  pieces  and  for  inspection.  The 
machine  vision  system  is  integrated  in  a  workcell  together  with  the  robot  manipulator  forming  a  complete 
robot  workcell. 

Machine  vision  can  be  used  in  conjimction  with  other  feedback  techniques  to  reach  a  high  level  of 
versatility.  Pmnell  et  al.  [54]  described  the  use  of  force  feedback  and  robot  vision  for  robot  meat  cutting. 
A  stereo  system  is  used  to  guide  the  end-effector  cutter  while  the  force  feedback  determines  the  appropriate 
force/torque  to  be  applied  at  the  robot’s  joints.  Verghese  et  al.  [69]  presented  a  real  time  motion  tracking 
of  three- dimensioned  objects.  The  system  can  be  used  to  track  reindomly  or'ented  products  that  eire  being 
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transported  by  a  conveyor. 


1.3.2  The  Quality  Control  Problem 

One  of  the  important  applications  of  machine  vision  in  the  food  industry  is  quality  control  (QC).  Even 
though  QC  is  one  of  the  most  labor  demanding  tasks,  it  can  not  be  implemented  by  fixed-automation. 
Quality  control  processes  is  a  task  where  decisions  must  be  based  upon  sensor  readings.  Complex  image 
processing  is  required  to  extract  information  from  a  digital  image.  This  qualitative  analysis  can  be  made 
possible  by  the  use  of  computers. 

However,  the  use  of  machine  vision  techniques  has  just  begun  to  gain  acceptance  in  the  food  industry. 
The  reason  for  this  is  that  the  quality  control  of  food  products  demands  special  characteristics  that  are 
not  present  in  existing  applications  [37].  Gagliardi  et  al.  [27]  discussed  two  of  these  characteristics,  the 
color  and  texture  measurements. 

Currently  most  of  the  quality  control  processes  (QCP)  are  performed  by  human  inspectors.  In  a 
highly  automated  plant  the  QCP  is  one  of  the  most  labor  intensive  operations.  This  is  because  automatic 
machines  can  produce  a  large  number  of  items  per  day  imder  little  supervision,  however,  all  these  items 
need  to  undertake  one  or  more  inspections. 

This  high  volume  of  items  to  be  inspected  generates  a  new  problem  when  using  machine  vision  in  QCP. 
In  order  to  keep  a  reasonable  productivity  level,  the  inspection  station  must  h^lndle  large  amoimts  of  items 
in  a  short  time.  Machine  vision  systems  must  acquire  images,  extract  features  and  determine  the  o’ ality 
in  short  periods  of  time  (from  100ms  to  a  few  seconds). 

There  are  several  reasons  lor  which  it  would  be  advisable  to  use  robots  in  QCP.  Robots  cm  work  longer 
shifts  and  their  use  con  avoid  contamination  of  product  due  to  direct  contact  with  human  inspectors. 
However,  economical  analyses  have  shown  that  robotizing  regular  plants  can  be  expensive,  and  the  return 
on  investment  could  tcike  several  yeeirs  [65]. 

For  this  reason  it  is  very  important  to  be  careful  when  studying  robot  applications  in  QCP.  It  is 
required  to  make  use  of  low  cost  mechanisms  to  be  able  to  determined  whether  or  not  robotizing  is  feasible. 
Computer  simulations  eind  small  scale  prototyping  cire  two  of  the  relatively  inexpensive  techniques  for  robot 
feasibility  studies. 

1.4  Computer  Simulation  for  Feasibility  Studies 

Robotic  systems  may  have  a  large  impact  on  the  productivity  of  industrial  plemts,  for  this  reason,  it  is 
very  important  to  study  the  feasibility  of  applications  of  such  a  technology  before  axi  implementation  is 
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undertaken. 

Several  methods  could  be  used  to  reach  fined  conclusions.  For  instance,  small  pieces  of  the  system  can 
be  prototyped  so  that  experimentation  is  performed  directly  on  them.  However,  because  in  many  cases 
prototypes  do  not  represent  the  complete  system,  it  is  difficult  to  obtain  a  genered  result.  For  instance,  the 
perform2ince  of  a  assembly  pl2Lnt  can  not  be  completely  determined  if  a  prototype  of  a  single  line  is  built 
and  analyzed.  This  is  because  an  assembly  line  may  be  closely  coupled  with  all  the  other  lines  On  the 
other  hand,  building  a  complete  prototype  is  in  many  cases  expensive. 

Another  way  to  study  the  performance  of  a  system  is  by  computer  simulation.  Numericed  and  graphics 
simulations  represent  two  powerful  tools  for  system  feasibility  analysis.  The  results  obtained  from  a  sim¬ 
ulation  model  Ccin  then  be  extrapolated  to  the  rccd  system.  Magnani  [45]  presented  a  complete  discussion 
on  modeling  «ind  simidation  of  an  industrial  robot.  The  aspects  of  sensor  feedback  for  interaction  with  the 
environment  are  covered.  In  this  way  collision  detection  and  real-time  path  planing  are  performed.  Such 
results  are  proved  to  be  completely  applicable  to  the  real  robot. 

The  simulations  can  be  used  to  predict  malfunction  under  extreme  conditions.  Chang  et  al.  [13]  used 
a  system  model  to  evaluate  the  diagnosability  of  failure  knowledge  in  manufacturing  systems.  This  type 
of  simulation  permits  to  determine  the  weaknesses  and  limits  of  a  system  without  performing  dangerous 
experiments  on  the  real  system. 

The  process  of  building  a  computer  simulation  can  be  briefly  outlined  as  follows  [56]: 

•  A  numerical  model  of  the  simulated  system  is  built.  This  numerical  model  is  made  by  selecting  a 
set  of  Vciriables  that  best  describe  the  system  rind  determines  its  behavior.  The  variables  represent 
internal  and  external  parameters  of  the  system. 

•  Implement  a  computer  program  that  represents  these  variables  and  their  behavior. 

•  Then  the  program  is  used  to  simulate  the  system.  This  is  done  by  varying  the  set  of  variables  that 
represent  external  conditions  while  the  program  calculates  the  state  of  the  internal  variables. 

In  this  way  it  is  possible  to  monitor  and  an^llyze  the  system’s  behavior  imder  different  conditions. 

A  special  case  of  computer  simulation  is  graphics  simulation.  This  type  of  simulation  meikes  use  of  the 
reed-time  einimation  [68]  to  display  a  graphical  representation  of  the  model.  In  this  case  the  variables  that 
describe  the  system  are  the  physical  dimension  and  the  spatial  location.  Troncy  [66]  developed  a  graphical 
interactive  simidation  that  allows  the  user  to  progreim  the  robot  by  interacting  with  the  graphic  simulation. 
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1.5  Statement  of  Work 


Combat  Ration  Advan^  i  Manufacturing  Technology  Demonstration  (CRAMTD)  is  a  program  of  The 
Center  for  Advancea  Food  Technology  (CAFT)  at  Rutgers  The  State  University  of  New  Jersey.  CRAMTD 
is  sponsored  by  the  DoD-Defense  Logistics  Agency  and  a  number  of  industri^d  pjirtners.  CAFT  is  one  of 
the  N:w  Jersey  Commission  on  Science  and  Technology  Advanced  Centers.  One  of  the  goals  of  CRAMTD 
is  to  demonstrate  flexible  automation  technologies  in  the  manufacturing  of  combat  rations. 

One  of  the  short  term  projects  of  CRAMTD  is  STPll;  “Feasibility  of  Application  of  machine  vision  and 
robotics  in  Packaged  Food  Manufacturing”  (Principcd  Investigator  Dr  Stanley  Dimn  aind  the  Co-Principal 
Investigators  Dr.  Grigore  Burdea  and  Dr.  Kucin-Chong  Ting).  The  statement  of  work  of  STPll  is  outUned 
as  follows: 

1.  Scope:  The  scope  of  this  project  is  to: 

(a)  determine  the  specific  tasks  of  the  components  required, 

(b)  develop  concepts  for  component  design, 

(c)  perform  system  integration  by  computer  simulation, 

(d)  determine  system  performtince  by  computer  simulation, 

(e)  assess  the  cost  effectiveness  of  the  system, 

(f)  recommend  methodology  for  further  development. 

2.  Techniciil  Approach:  The  CRAMTD  engineers  will  explore  the  feasibility  of  system  integration  in¬ 
cluding  machine  vision  eind  robotics  applications  for  MRE  pouch  inspection.  In  the  case  of  machine 
vision,  defective  pouches  will  be  collected  from  contractors,  who  will  be  asked  to  identify  and  clas¬ 
sify  the  nature  of  the  defects.  These  defective  pouches  will  be  subjected  to  several  machine  vision 
examination  techniques,  such  as  the  use  of  colored  light,  the  use  of  polarized  light,  the  use  of  image 
processing  algorithms  and  the  use  of  range  image  techniques. 

The  possible  robotic  workcell  layouts  and  material  handling  scenarios  will  be  studied  in  concert  with 
the  machine  vision  technique  developed.  The  material  manipulation  actions  required  to  accomphsh 
the  pouch  inspection  will  be  categorized  into  motions  which  need  to  be  done  by  fixed  automation 
and  flexible  automation  concepts. 

Engineering  peircimeters  which  affect  the  performance  of  the  workcell  will  be  identified.  Computer 
modeling  techniques  will  be  used  to  develop  component  numerical  models.  The  component  models 
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will  be  integrated  to  form  a  system  model.  Design  parameters  will  be  varied  and  entered  to  the 
system’s  model  to  simulate  the  performance  of  the  workceU  under  various  conditions.  The  simulation 
process  will  facilitate  the  development  of  a  workable  system. 

Engineering  economic  analyses  will  be  performed  on  the  workable  system  to  compare  the  cost- 
effectiveness  of  different  systems. 

3.  Program  Schedule:  This  program  will  be  conducted  in  a  single  phase  consisting  of  5  tasks  and  a  Final 
Report  as  shown. 

(a)  Review  Current  Design:  Consists  of  a  review  with  combat  ration  mtinufacturers  and  CRAMTD 
personnel.  This  review  will  identify  the  requirements  for  robot  and  vision  workceUs  for  MRE 
pouch  inspection. 

(b)  Identify  Candidate  Components:  In  this  task,  potential  design  strategies  to  satisfy  the  require¬ 
ments  will  be  established. 

(c)  Develop  Component  cind  System  Numerical  Models:  This  task  will  involve  the  development  of 
numerical  models  for  workceli  components  and  the  entire  system. 

(d)  Computer  Simulation  to  Identify  Workable  System  Design:  Using  the  models  developed  a  series 
of  design  conditions  will  be  systematically  studied. 

(e)  Economic  Analysis:  Under  this  task,  evcduation  of  the  system’s  cost-effectiveness  shall  be  per¬ 
formed.  The  result  shall  be  sufficient  to  recommend  whether  or  not  further  coiTimercial  devel¬ 
opment  is  warremted. 

(f)  Final  Report. 

The  material  developed  in  this  report  covers  th"  topics  included  in  the  STPll  progrcim  schedule. 
Attention  is  focused  on  the  computer  workcell  simulation,  the  robotic  manipulation  problem  cmd  the 
macliine  vision  problem.  The  economical  implications  are  cons.,'3red  in  Section  6. 

1.6  Report  Outline 

The  first  part  of  Section  Two  presents  the  problem  of  numerical  simulation  of  the  QCP  and  the  relevant 
issues  of  the  system  modeling.  The  Section  starts  with  a  definition  of  the  basic  units  that  form  a  typical 
quality  control  plant.  A  model  for  defects  and  their  detection  dining  inspection  is  presented.  The  inspection 
time  and  inspection  accuracy  are  discussed  as  a  fimction  of  the  item’s  quality. 
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The  second  peirt  of  Section  Two  describes  the  software  developed  for  numerical  simulation  (the  Sys¬ 
tem  Simulation  Object  Oriented  Libr2iry  “SOL”).  A  simple  simulation  program  using  and  SOL  is 

explained.  It  shows  how  SOL  facilitates  the  implementation  of  emy  QCP  simulation  by  just  mapping  real 
entities  2ind  its  intercoimections  to  software  objects. 

The  last  part  of  Section  Two  presents  the  simulation  of  a  real  plcint.  The  simulation  is  created  with 
data  gathered  directly  from  three  rccd  food  processing  plants.  An  alternative  simulation  model  using 
robotic  inspector  characteristics  is  developed  as  well.  The  two  simulations  are  compared  cind  conclusions 
on  automatic  vs.  manusd  plants  are  obtciined. 

Section  Three  describes  the  interactive  graphics  simulator  IGAS.  This  simrdator  is  a  complement  to 
the  numerical  simiilation.  As  opposed  to  quantitative  characteristics  (numerical  Vcdues),  IGAS  permits  the 
exploration  of  queilitative  characteristics  (physical  shape  and  location)  of  QCP  models  Both,  a  human 
based  and  a  robot-based  workceU  Me  simulated  using  IGAS.  The  user  can  interact  with  the  graphics 
simulation  by  means  of  a  track-ball  device  which  allows  the  user  to  “fly”  through  the  simulated  workcell. 

In  Section  Four  the  problem  of  pouch  manipulation  is  presented.  The  pouch  handling  is  separated 
into  three  zones.  The  characteristics  and  manipulation  needs  of  each  zone  are  analyzed.  Two  models  that 
satisfy  the  handling  requirements  are  presented.  The  models  differ  mainly  by  their  level  of  automation. 
The  first  uses  a  commercieil  industrial  robot  while  the  other  uses  a  custom  designed  manipulator.  In  both 
CMes,  the  genereil  design  and  control  requirements  are  explained. 

Section  Five  is  a  discussion  of  the  machine  vision  technique  developed  to  detect  the  defective  pouches. 
The  physical  principxes  and  a  mathematical  model  for  light  reflection  that  is  the  fundamental  basis  for 
the  inspection  system  is  presented.  The  limitations  eind  capacities  of  this  new  approach  to  inspection  are 
explored,  concluding  with  a  model  of  how  these  fundamental  physical  principles  can  be  used  to  inspect 
MRE  pouches  automatically.  In  addition,  several  edtemative  techniques  for  defect  detection  are  suggested. 

Section  Six  is  a  summary  of  our  economic  einailysis  of  the  two  proposed  automation  systems.  The  section 
is  divided  into  two  parts:  The  first  part  is  an  overview  of  the  computer  based  economic  analysis  system 
that  was  chosen  for  this  phase.  The  second  part  of  section  6  is  a  simunMy  of  the  results  for  workcell  versus 
hum^ln  costs,  equivalent  capacity,  feasibility  an2dysis  and  parametric  analysis. 

Section  Seven  presents  the  conclusion  of  this  research.  A  selection  of  the  code  used  to  create  the 
simulation  is  presented  in  Appendix  A.  The  same  appendix  includes  the  code  for  the  classes  definitions  of 
the  SOL  library. 
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1.7  Conclusions  and  Recommendations 


In  sununary,  this  report  describes  a  machine  vision  system  for  inspecting  MRE  pouches  that  is  based  on 
fimdcunentcil  physical  characteristics  of  the  incident  radiation,  the  surfaces  imaged  and  their  interaction 
The  optimizations  described  herein  have  not  been  used  before  in  the  food  inspection  industry. 

Based  on  this  theoretical  model,  a  proposed  machine  vision  paradigm  was  established  and  a  prototype 
embodiment  of  this  pciradigm  was  constructed.  The  prototype  built  for  experimentation  and  verification 
is  described  in  this  report.  The  results  of  defect  detection  experiments  showing  its  diagnostic  performance 
establishes  technical  feasibility  of  the  imaging  paradigm. 

Two  approaches  for  robotic  transport  through  the  inspection  workcell  were  considered.  The  first  is  a 
fixed  or  hard  automation  system  and  the  second  is  a  flexible  automation  system.  Both  of  these  approaches 
were  simulated  and  the  result  are  contained  herein.  Based  on  the  simulations  and  the  economic  analysis 
of  each,  we  find  that  the  fixed  or  hard  automation  system  is  the  more  economically  feasible  of  the  two 
approaches.  In  quantities,  the  system  has  a  simple  payback  in  imder  two  years  with  a  significantly  high 
return  on  investment. 

Thus,  we  conclude  that  automatic  inspection  of  MRE  pouches  is  both  technically  and  economically 
feasible. 
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2  Numerical  Simulation 

2.1  Introduction 

This  section  is  divided  into  three  parts.  In  the  first  part,  the  QCP  is  decomposed  into  several  simple  inde¬ 
pendent  components.  Each  component  is  discussed  in  detcul.  Based  on  the  components  maun  characteristics 
generic  numericed  models  are  proposed. 

In  the  second  part,  the  System-simulation  Object-oriented  Library  “SOL”  is  presented.  SOL  is  a  libr^y 
of  C-f-f  classes  (objects)  used  to  build  nvimericcd  simulations  of  QCP.  Each  one  of  the  SOL’s  classes  is  a 
implementation  of  the  models  of  QCP’s  components.  An  illustrative  exeimple  of  a  simulation  of  a 
single-line  inspection  workceU  using  SOL  is  discussed. 

In  the  last  part,  a  simulation  for  an  existing  plant  is  built.  The  results  of  the  simulation  Me  compared 
with  real  data  collected  from  industry.  Finally,  the  simulator  is  used  to  build  a  plant  with  the  characteristics 
and  parameters  of  a  robotized  plant.  The  topology  and  some  controllable  pMameters  are  varied  so  that  the 
robotized  plant  matches  the  performance  of  the  human-operated  plant.  The  two  simulations  Me  compMed 
and  conclusions  Me  given. 

2.2  System  Model 

Controlling  the  quality  of  products  is,  in  general,  a  complex  task  which  involves  many  steps.  However, 
t3rpic2il  quality  control  systems  can  be  decomposed  into  several  simpler  miits  that  perform  specific  tasks. 
Such  tasks  cm  be  easily  identified,  and  computer  models  can  be  built  for  them. 

A  description  of  the  basic  components  that  form  a  generic  quality  control  inspection  workcell  will  now 
be  presented.  At  this  level  of  the  model  no  distinctions  Me  made  of  whether  or  not  some  components 
Me  implemented  by  hximan  operators  or  automatic  machines.  This  generality  permits  the  model  i  >  be 
extrapolated  from  one  extreme  to  the  other  by  just  chmging  some  performance  chMacteristics. 

This  extrapolation  from  humm-operator  to  robot-operator  will  vahdate  the  feasibihty  of  the  application 
of  robotic  technology  in  a  quality  control  process. 

2.2.1  Workcell  Component  Modeling 

Here,  the  basic  components  of  a  generic  quality  control  process  are  defined.  The  basic  components  will  be 
kept  as  general  and  simple  as  possible,  so  that,  numerical  models  can  be  easily  developed.  The  idea  behind 
this  simplicity  is  not  to  ignore  complexity,  but  on  the  contrMy,  to  have  the  building  block  required  to  achieve 
the  implementation  of  more  complex  models.  Some  of  the  components’  chMacteristics  will  be  defined  in 
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terms  of  some  general  variables  or  data  structures  that  Ccin  be  implemented  in  any  computer  language. 
The  reason  for  this  descriptive  selection  is  to  keep  the  model  aimed  towards  a  numerical  simulation. 

Item:  The  parts  whose  quzdity  is  to  be  controlled  will  be  called  items.  An  item  can  be  of  any  nature 
and  is  considered  to  be  the  minimum  unit  of  production  to  be  manipulated.  The  main  ch^acteristic  that 
is  relevcmt  for  simulation  purposes  is  the  item's  quality  that  Ccin  be  represented  by  a  real  value  as  discussed 
later.  In  addition  to  quality  characteristics,  items  carry  an  identification  data  structure  that  could  contain 
information  such  as  serial  number,  date  of  manufacturing,  contents,  inspection  results,  etc. 

Source:  At  the  beginning  of  every  quality  control  workcell  there  are  source  imits.  Source  units  emit 
all  the  items  that  flow  through  the  inspection  process.  In  simulation,  sources  Eire  in  chairge  of  creating 
the  data  for  the  items  emitted,  including  their  defects.  In  a  resd  situation,  a  source  could  be  one  of  msmy 
different  things:  contEiiners,  warehouses,  process  emitting  items,  etc.  However,  for  simulation  reasons  these 
distinct  natures  Eire  irrelevEint.  What  is  important  during  simtilation  when  dealing  with  source  units  Eire 
their  output  rate  -meEisured  in  items /sec-  Eind  their  quEility  distribution. 

The  output  rate  of  smy  source  unit  is  in  generEil  a  normEil  distributed  random  VEiriable  -chE^acterized 
by  its  meEin  value  Eind  dispersion-.  The  quality  distribution  says  what  is  the  probability  of  Ein  emitted 
item  to  have  a  given  quality.  A  broken  source  emits  a  larger  number  of  defective  items  than  a  regular 
one.  The  functionsdity  condition  of  a  source  component  can  be  modeled  by  a  uniform  distributed  vEiriable 
representing  the  reliability  of  the  given  source.  A  threshold  value  is  selected  so  that  when  the  reliability 
variable  is  below  (or  over)  this  value  the  source  is  considered  to  be  broken. 

Buffer:  Dming  the  path  followed  by  the  items  in  a  QCP,  the  items  may  have  to  be  temporEirily  stored 
or  stacked  in  special  places.  This  is  because  some  parts  of  the  process  could  take  longer  than  others  Eind 
the  items  coming  from  a  previous  faster  process  have  to  be  temporarily  accmnulated.  These  places  Eire 
called  bujfer  units  Eind  sue  chEiracterized  by  their  capacity  measured  in  items.  For  the  simulation  models  in 
discussion,  buffers  are  considered  to  be  passive  elements.  That  is,  the  only  way  to  access  them  is  tlirough 
transfer  units  (discussed  later),  which  access  the  buffer  hy  putting  in  or  taking  out  items.  In  other  words, 
buffers  by  themselves  are  not  able  to  get  or  send  items  from  or  to  other  units. 

Inspector:  The  process  of  controlling  the  quality  of  the  items  is  called  “inspection”.  The  unit  or 
element  in  charge  of  inspection  is  the  inspector  tmit.  Inspectors  look  for  a  specific  set  of  defects  that  has 
been  established,  and  reject  or  mark  items  that  do  not  satisfy  quality  standards.  The  time  required  for 
Ein  inspector  to  inspect  completely  an  item  is  called  inspection  time.  The  success  rate  is  the  percentage 
of  times  that  an  inspector  is  able  to  assert  the  correct  quality  of  the  items.  The  inspection  time  Eind 
the  success  rate  chEiracterize  completely  the  inspector’s  performEince.  Those  VEiriables  that  depend  on  the 
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quality  characteristics  of  the  particulcir  item  being  inspected  are  discussed  later. 

There  are  several  special  classes  of  inspectors.  Some  inspectors  have  the  special  function  of  tabulating 
data  from  rejected  items  or  supervising  the  work  of  a  regular  inspector.  The  cheuacterization  of  special 
inspectors  could  require  special  variables  different  from  the  inspection  time  or  success  rate.  Those  cases 
are  discussed  independently  and  tue  considered  to  be  exceptions.  All  inspectors  access  the  items  via  a 
transportation  unit  that  brings/tcikes  the  items  from/to  the  inspector. 

Conveyor;  The  transportation  of  the  item  through  the  process  is  accomplished  by  meeins  o  uts: 

the  conveyors  and  the  transfers.  Conveyors  carry  items  for  a  fixed  path  at  a  constant  or  variable  ,  d.  In 
general,  the  conveyor's  path  is  such  that  inspectors  or  other  units  can  take  items  and  put  them  back  in  the 
conveyor  after  some  processes  have  been  performed  (marking,  registering,  inspecting,  etc).  There  are  two 
parameters  that  chciracterize  conveyors,  the  capacity  and  the  transport  time.  The  capacity  is  considered 
to  be  fixed,  however,  the  transport  time  is  a  function  of  the  conveyor's  speed  and  could  change  during  the 
process.  Some  inspectors  and  other  specied  units  have  the  ability  of  controlling  the  speed  of  the  conveyor  if 
required.  Conveyors  could  be  used  to  move  items  between  units,  but  this  case  occurs  only  when  no  specizJ 
manipulation  at  the  front  end  of  the  conveyor  is  required,  for  example:  stacking,  palletizing,  depalletizing, 
etc. 

Transfer:  Transfer  units  are  in  cheirge  of  moving  items  between  units.  Transfer  units  can  execute 
simple  organizational  tasks  such  as  stacking,  pedletizing,  depalletizing  before  or  after  the  items  are  moved. 
Even  activities  like  air-drying  before  palletizing  (or  depalletizing)  can  be  assigned  to  transfer  units.  For 
simulation,  the  spatial  organization  of  items  into  containers  or  stacks  is  not  relevant,  however,  what  is 
tciken  into  account  is  the  time  required  for  such  a  msmipulation.  That  is  why  the  main  characteristic  of 
a  transfer  unit  is  the  time  required  to  complete  the  transfer  process  and  be  ready  to  receive  or  to  take 
new  items.  This  time  is  cedled  transfer  time.  The  capacity  of  a  transfer  unit  (transfer  capacity)  is  adso 
important  as  they  could  move  more  thain  one  item  at  a  time. 

Another  importcint  task  performed  by  transfer  units  is  selecting  the  path  of  inspected  items.  As 
mentioned  before,  inspectors  mark  or  reject  items  with  low  quality.  When  the  items  have  been  just  meirked 
the  transfer  unit  that  follows  the  inspection  will  be  in  charge  of  discarding  the  bad  item.  For  simulation, 
transfer  units  have  an  extra  feature.  Because  they  are  used  to  interface  two  or  more  units,  all  the  time 
delay  cam  be  easily  lumped  into  the  transfer  units. 

Accepted  &  Discarded:  Once  an  item  has  gone  through  the  inspection  process  it  may  be  classified 
as  either  accepted  or  discmded.  The  final  destination  of  the  accepted  items  is  called  the  accepted  unit.  In 
real  cases  these  are  lairge  warehouses  or  large  buffers  that  store  items  before  they  go  through  a  new  process 
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(if  required).  For  the  simulation,  the  accepted  units  cue  just  unlimited  capacity  buffers  that  perform  some 
statistical  analyses  using  the  accumulated  data  of  the  accepted  items.  The  coimterpart  of  the  accepted 
xmit  is  the  discarded  unit  that  receives  the  bad  items  and  performs  equivalent  statistical  analyses. 

Processes:  Manufacturing  products  is,  in  general,  a  multi-stages  process.  In  many  cases  produced 
items  have  to  imdertake  quedity  control  inspection  between  processes.  In  simple  simulated  cases  process 
units  can  be  considered  just  as  delays  in  an  item’s  path.  However,  a  process  could  eventually  change  the 
item's  quality.  Furthermore,  new  types  of  defects  have  to  be  added  or  deleted  to  the  defect  table  eifter 
items  have  passed  through  some  processing.  If  the  complete  quality  control  workcell  is  to  be  considered, 
processes  have  to  be  included  during  simulation.  For  the  simulations  discussed  in  this  report,  processes  are 
considered  to  be  large  buffers  with  large  delays. 

2.2.2  Modeling  Defects 

In  general  defects  czin  be  of  any  nature,  however,  for  specific  products  there  exists  a  set  of  specific  types  of 
defects.  In  such  cases,  defects  are  weU  known  and  their  characteristics  are  tabulated  and  established  before 
any  quality  control  is  undertaken.  Inspectors  are  trained  (or  programmed)  to  discover  such  defects  and  to 
be  able  to  decide  quickly  and  with  high  accuracy  whether  or  not  the  quality  of  the  items  is  adequate. 

To  model  defects  in  simulation  it  is  not  required  to  deal  with  the  defect’s  intrinsic  nature.  What 
is  important  is  to  know  its  statistical  behavior.  Defect  probability  distributions  or  defect  rates  are  two 
statistical  characteristics  that  can  be  obt2iined  from  experimental  sampling  or  just  from  data  coming  out 
of  real  QCP  pl2mts.  Stemdard  simulation  techniques  axe  used  in  this  modeling  [56]. 

To  characterize  quantitatively  the  set  of  defects  D  =  {di,d2,  a  quality  level  qi  is  assigned 

to  every  defect  di  .  In  this  way,  the  overall  quality  of  an  item  is  given  by  the  set  of  quality  levels 
Q  —  {9I)  92)  -M  corresponding  to  the  set  of  defects  D  .  The  defects  considered  here  cire  asstuned  not 
to  be  correlated,  hence  the  quality  levels  qi  and  qj  ;  qi,qj  €  Q  are  independent. 

The  simplest  model  for  quality  levels  q  is  to  assiune  two  (bineu^y)  values  of  quality,  good  qgood  or  bad 
9(«i(i)  each  one  with  a  given  probability.  To  generate  a  variable  with  such  characteristics  is  just  a  matter  of 
using  a  uniform  distributed  variable  it  and  a  threshold  value  Uthreahaid  (Figure  la).  The  veilue  of  q  is  then 
defined  as: 

{Qgood  i/  U  <  Uthreahold 
%ad  if  ^  ^  ^threshold 

such  a  binciry  distribution  is  shown  in  Figure  lb. 

However,  for  a  more  realistic  item-inspector  interaction  (discussed  in  the  next  subsection)  it  is  better 
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Figure  1:  Binary  non-uniform  distributed  variable 


Quality  Level  Probability  Distribution 


P(q,) 


Threshold  For  Rejection  (TFR) 


1 

(Perfect-BadI 


Good  Perfect 


(Quality  Levels) 


Figure  2:  Quality  distribution 


to  allow  a  larger  range  of  quality  levels.  The  following  paragraphs  presents  a  description  of  the  qurdity 
modeling  used  for  the  simulations. 

Figure  2  shows  the  probability  distribution  of  the  quality  level  qi  associated  with  defect  i  .  It  is  just 
a  uniform  distribution  on  the  interval  [Bad,  Perfect]  .  The  TFRi  (Threshold  For  Rejection)  level  is  the 
threshold  level  for  quality  control  assigned  to  defect  i  .  The  quedity  control  process  cjin  be  described  as 
follows:  Items  that  have  any  of  the  quedity  levels  qi\i  =  falling  in  the  interval  [Bad,TFRi] 

must  be  rejected.  Hence,  items  with  q,  in  the  interval  [TFRi,  Per  feet]  are  considered  to  satisfy  the 
quality  standards  and  are  accepted.  (Note  how  the  last  interval  has  an  open  boundary  in  the  vzdue  TRFi 
to  avoid  conflicts). 

For  numerical  simulations  the  interval  [Bad,  Perfect]  is  normalized  as  the  interval  [0, 1]  hence 
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0  <  TFR  <  1  .  The  qi;  i  =  1,2,...,  N  are  generated  by  means  of  a  uniform  pseudo-random  generator.  The 
TFRi;  i  =  l,2,3...,iV  are  fixed  before  the  simulation.  Their  values  are  chosen  so  that  TFRj  *  100  =  Pj  , 
where  Pj  is  the  percentage  of  defective  items  with  defect  j  .  During  simulation,  when  an  item  m  is 
to  be  issued  from  a  source  imit,  the  set  of  quahties  Qm  associated  to  item  i  is  created  via  a  raindom 
generator.  This  N  component  vector  is  attached  to  the  data  structure  of  item  m  . 

What  follows  is  a  description  of  the  interaction  of  the  items  with  the  inspectors.  It  will  be  shown  how 
the  qucdity  levels  influence  the  inspection  results  qualitatively  and  quantitatively. 

2.2.3  Modeling  the  Inspection  Process 

The  emphasis  will  now  shift  to  the  components  of  the  inspection  process.  The  required  definitions  are 
introduced  as  well  as  the  description  of  how  the  inspection  is  carried  out.  The  two  main  aspects  of  the 
inspection  performance  are  covered.  The  inspection  time  and  the  inspection  accuracy.  The  techniques 
used  here  to  correlate  stochastic  variables  zure  described  in  [56]. 

The  performcince  of  any  inspector  could  be  affected  by  the  quality  level  of  the  item  being  inspected.  If 
the  quality  is  so  good  that  the  item  could  be  considered  “perfect”,  the  inspector  does  not  have  to  spend 
extra  time  deciding  or  doing  further  inspection.  If  the  quality  is  very  bad,  the  inspector  can  reject  very 
quickly  the  item.  However  if  the  item’s  queility  is  such  that  it  is  close  to  the  threshold  for  rejection  TFR, 
ciny  inspector  could  spend  extra  time  trying  to  determine  the  correct  quality  of  the  item.  Furthermore,  in 
this  particular  situation  inspectors  are  most  likely  to  misjudge.  In  other  words,  items  with  a  quality  level 
in  the  neighborhood  of  the  threshold  level  TFR  eure  critical  for  inspection.  For  these  items  the  inspection 
time  increases  and  the  inspector  accuracy  decreases. 


2.2.4  Inspection  Time 

The  toted  inspection  time  of  inspector  k  required  to  inspect  one  item  is  denoted  as  t^,  and  is  defined  as: 

(2) 


N 


where  tki  is  the  time  used  by  inspector  k  to  inspect  defect  i  alone.  The  sum  goes  over  edl  the  set  of  defects. 

tki  is  modeled  by  means  of  a  normally  distributed  bounded  variable  with  probability  P{tki)  shown  in 
Figure  3  and  given  by: 

P{tki)  =  ae  V  (3) 

for  <  tki  <  and: 


P{tki)  =  0 


(4) 
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for  tki  <  or  tfc,  >  The  latter  bounds  the  variable  t^i  to  avoid  unrealistic  values. 

To  implement  the  correlation  between  item  quality  and  inspection  time,  the  mean  is  made  a 
function  of  the  quality  level  qi  of  the  particular  item  being  inspected.  This  relation  is  shown  m  Figme  4 
and  is  given  by: 


_  f  if  Bad  <qi<  TFRu 

tki  =  <  (5) 

i  T™  -  if  TFRi  <  qi  <  Perfect 

The  values  and  i  =  1,2,  ...,N  are  constant  pcirameter  of  the  inspector  k. 

The  m£iximmn  average  inspection  time  tki  =  T^i^^  occurs  when  the  quality  of  the  item  qi  is  equal  to  the 
threshold  for  rejection  TFRi. 

2.2.5  Inspection  Error 

The  inspection  process  is  a  stochastic  process.  No  inspector  is  able  to  determine  with  total  accuracy  the 
real  quality  level  g,  corresponding  to  defect  i  of  a  given  item.  Inspector  k  judges  the  quality  qi  by 
doing  Ein  estimate  qki  which  is  compared  agzunst  the  TFRi  ■  Inspectors  are  trained  or  programmed  to 
minimize  the  error  |?i  —  ?fct|  •  However,  as  mentioned  before,  the  estimation  error  could  increase  as  the 
quality  level  approaches  the  critical  value  TFRi  ■ 

A  good  simulated  model  is  obtained  when  the  estimate  qki  is  tciken  as  a  normal  distributed  rcindom 
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variable  with  mean  gli  and  stcindard  deviation  aki  .  In  this  case  the  mean  value  q'ki  is  the  reed  quality 
level  qi  ;  qli  =  qi  .  So  the  estimate  is  normzJly  distributed  around  the  real  value. 

To  model  the  fact  that  the  estimation  error  depends  on  the  value  of  qi  the  standaud  deviation  (Xki  is 
made  a  function  of  qi  .  A  similar  function  to  the  one  used  for  the  mean  inspection  time  is  used  here.  This 
ftmction  is  shown  in  Figure  5  and  is  given  by: 


(Tki  =  < 


TFR 


<r + 

I  -TFR  (v-TFRi)  t -TFR  -Perfect^ 

I  ) 


if  Bad  <  qi  <  TFRi 
if  TFRi  <  qi  <  Perfect 


(6) 


In  this  way  the  accuracy  approaches  a  minim\im  when  the  quality  level  qi  is  close  to  the  TFRi  • 
This  happens  because  the  dispersion  of  the  estimation  qid  given  by  cr^j  is  majdmized.  In  this  case  the 
estimation  error  \qi  —  gitil  is  most  likely  to  take  larger  values. 

During  simulation,  aifter  the  inspection,  the  set  of  estimated  quality  values  given  by  (j  =  {91, 92,  97v} 

is  attached  to  the  data  structure  of  every  item.  In  the  accepted  or  discarded  units,  the  values  of  Q  and  Q 
are  compared,  cind  the  statistical  accuracy  of  the  overall  process  is  accumulated. 

2.3  Numerical  Simulation  Software 

What  follows  is  a  description  of  the  System  Sim^llation  Object-Oriented  Library  tool  c2illed  SOL.  SOL  was 
created  to  facilitate  the  task  of  implementing  numerical  simulations  of  QCP.  SOL  is  a  library  of  classes 
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Figure  5:  Standard  deviation  of  estimated  quality:  Inspector  k,  defect  i. 


implemented  in  C++  [23,  60].  These  classes  contain  models  of  the  basic  units  as  described  previously. 
Special  functions  were  added  to  the  library  which  help  to  define  the  QCP  cind  carry  out  the  simulation. 

When  implementing  a  simulation,  the  classes  are  used  to  define  objects  that  behave  numericaJly  as  the 
basic  units  described  earlier.  Objects  cire  closed  structures  that  conteun  data  and  internal  functions  ceilled 
methods.  The  Object’s  data  are  the  variables  required  to  model  the  internal  state  of  a  unit.  Methods 
access  and  modify  the  internal  data,  hence,  methods  model  the  dynamic  behavior  of  the  basic  unit.  Some 
special  functions  are  used  to  interconnect  the  objects  in  order  to  reassemble  the  connectivity  of  simulated 
processes.  Other  specicd  fimctions  ^lre  used  to  change  default  peirameters  or  execute  the  simulation. 

Rather  thain  discussing  the  intrinsic  detmls  of  SOL’s  classes,  their  use  is  described.  An  example  is 
presented  as  am  illustration.  This  simple  case  is  zdso  an  introduction  for  the  more  robust  application 
presented  later. 

2.3.1  The  System  Simulation  Library  Tool  “SOL” 

The  implementation  of  a  simulation  using  SOL  involves  the  following  steps: 

1.  Include  the  header  files  sol.h,  bclasses.h,  sim.h 

2.  Define  aU  the  objects  representing  the  basic  units  that  are  required  to  assemble  the  plcint.  This  is 
done  by  using  the  predefined  classes  in  SOL. 


■ 
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3.  Assign  identification  numbers  to  the  objects  and  redefine  the  default  characteristics.  This  is  done 
by  using  the  SOL  function  install.  All  units  that  are  going  to  be  used  durmg  simulation  must  be 
instjJled. 

4.  Define  the  connectivity  of  the  plant.  This  is  done  by  means  of  the  SOL  function  connect.  Connect 
establishes  a  direct  unidirectional  interaction  connection  between  two  units. 

5.  Configure  basic  parameters.  For  example:  the  number  of  defects  with  the  function  defect(N)  and  the 
TFRi  by  assigning  new  values  to  the  array  TFR[]. 

6.  Establish  the  initial,  final  and  step  simvilation  times  as  well  as  the  progress  report  formats.  This  is 
done  by  means  of  the  function;  simuLtime,  and  prog.nport. 

7.  Execute  the  simulation  using  the  function  simulate.  This  fimction  is  also  capable  of  performmg  step 

6. 

Many  other  functions  furnished  by  the  user  can  be  added  to  SOL  allowing  step  by  step  simulation,  trap 
operation  or  other  customized  feature  wanted. 

To  compile  the  final  executable  code  the  files  bclasses.c  and  sim.c  could  be  compiled  (C++  compiler 
must  be  used)  together  with  the  simulator  file.  For  instance,  if  the  simulator  is  called  simul.c  the  compi¬ 
lation  command  is; 

c+-h  -o  simul  simul.c  sim.c  bclases.c 

Otherwise,  the  simulator  cam  be  compile  just  by  linking  the  source’s  object  file  with  the  library  sol  as 
follows: 

c-f-t-  -o  simul  simul.c  -Isol 

Details  on  compiler  commeinds  can  be  found  in  the  mein-pages  of  Unix  [2].  A  simple  but  robust 
application  of  SOL  is  now  given. 

2.3.2  Example:  Small  Plant  Simulation 

The  schematic  for  the  plcint  is  shown  in  Figure  6  This  case  is  a  simple  quality  control  process  that  medces 
use  of  all  the  basic  units  defined  before.  The  process  is  a  single  inspection  line.  After  inspection,  the 
accepted  items  go  to  the  buffer^  unit  then  through  the  transfer-A  unit  eind  end  in  the  accepted  unit. 
Rejected  items  do  the  path  starting  at  the  bufferS  unit  ^lnd  ending  in  the  discarded  unit. 

The  C++  code  for  this  workcell  using  SOL  is  as  follows: 
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iinclude  <stdio.h> 

#include  <streain.h> 
iinclude  <math.h> 
iinclude  <stdlib.h> 
iinclude  <time.h> 
iinclude  <ctype.h> 

//  STEP  1  *********************************»***m*************m* 

iinclude  "sol.h" 
iinclude  "bclasses.h" 
iinclude  "sim.h" 

mainO 

{ 

/  /  STEP  2 

//  Define  the  units  (Objects)  that  are  going  to  be  used.  // 

//  change  default  parameters  (as  listed  here) 

Source  source(l) ;  //issue  rate  1  item/sec 

Buffer  buffer(lOO);  //capacity  100  item 

Buffer  buffer.A(lOO) ;  //capacity  100  item 

Buffer  buff er_B(20) ;  //capacity  20  item 

Transfer  trzoisferd  ,  1)  ;  //capacity  1  item,  sped:  Isec/item 

Transfer  transf _A(1 ,3) ;  //capacity  1  item,  sped;  3sec/item 

Transfer  transf .B(l ,3) ;  //capacity  1  item,  sped;  3sec/item 

Conveyor  conveyor(4);  //capacity  4  item 

Inspector  inspector(l) ;  //insp.  time  1  sec. 

Accepted  accepted;  //None. 

Discarded  discarded  ;  //None. 

//  STEP  3 

//  Install  the  unit  and  assign  a  ID  number  // 

instalK  Asource,  10  ); 

instalK  Abuffer,  20  ); 

instalK  Abuffer.A,  21  ); 

instalK  Abuffer.B,  22  ); 

instalK  ^transfer,  30  ); 

instalK  Atransf.A,  31  ); 

instalK  fttransf_B,  32  ); 

instalK  ftconveyor,  40  ); 

instalK  Ainspector,  50  ); 

instalK  Aaccepted,  60  ); 

instalK  Adiscarded,  70  )  ; 

//  STEP  4  «*****«**i|>«***>k****’XiK>t>***’ti«4<***4i>t>«4i*4c*4i«*««***4i**** 
//  Connect  the  units  // 

connect (  10,  20  );  //  source  ->  buffer 

connect (  20,  30  ) ;  //  buffer  ->  transfer 

connect (  30,  40  );  //  tremsfer  ->  conveyor 

connect (  40,  50  );  //  conveyor  ->  inspector 

connect (  50,  22,  GOOD  );  //  inspector  ->  buffer. A 

connect (  50,  21,  BAD  );  //  inspector  ->  buffer.B 

connect (  21,  31  );  //  buffer.A  ->  transfer. A 

connect (  22,  32  );  //  buffer. B  ->  transf er.B 

connect (  31,  60  );  //  treuisfer.A  ->  accepted 


connect (  32.  70  ); 


//  tramsler.B  ->  discarded 


//  STEP  5  ******m*m****************»i***mmm***mm*****m******** 
defects (4) ; 

TFR[0]  =  0.01  //  1'/,  defective 
TFR[1]  =  0.01  //  17.  defective 
TFRC2]  =  0.01  //  1'/,  defective 
TFR[3]  =  0.01  //  1*/,  defective 

//  STEP  6  euid  7  ****«**********«***3tE«»iti*«««*»«iti«««**itc**:«c*ittiii** 
//  Ceirry  out  the  simulation  from  time  0.0s  to  time  30.0s 
//  in  steps  of  1.0s. 
simulate (  28800.0,  1,  FALSE  ); 


The  parameters  of  the  fimction  simulate  specify  the  final  time,  simulation  step  and  result  report  format. 
The  time  used  in  this  code  corresponds  to  one  complete  working  day  I8h  shift).  In  this  case  the  default 
parameters  are  chzinged  during  the  insteintiation  of  the  units  (definition  of  the  objects).  Note  that  when 
connecting  the  inspector,  a  third  parameter  has  been  used.  This  pairameter  determines  the  kind  of  items 
that  can  go  through  the  connection.  In  this  case  buffer^  takes  the  accepted  items  while  buffer^  takes 
discarded  items. 

The  peirameter  FALSE  in  the  function  simulate  indicates  that  the  program  shoidd  report  results  at  the 
end  of  the  simulation  and  not  at  every  step. 

The  results  report  is  presented  in  the  default  way  as  follows: 


***  QCP  Simulation  using  SOL  VI. 0  *** 


Unit  Type  | 

ID 

1  Input  1 

Output  1 

1  Kept  1 

Source 

10 

- 

14500 

- 

Buffer 

20 

14500 

14400 

100 

Buffer 

21 

7180 

7180 

0 

Buffer 

22 

7220 

7200 

20 

Conveyor 

30 

14400 

14400 

0 

Conveyor 

31 

7180 

7179 

1 

Conveyor 

32 

7200 

7199 

1 

Inspector 

40 

14400 

14400 

0 

Transfer 

50 

14400 

14400 

0 

Accepted 

60 

13768 

- 

13768 

Discarded 

70 

576 

- 

576 

27 


50  14400/14397  (99.97*/.) 


Source 

biall 

ID  1 

stalls  1 

10 

300 

As  shown  in  the  previous  example,  implementation  of  QCP  simulations  is  straightforward  if  SOL  is 
used.  The  previous  simulation  was  run  m  a  SUN  4/380  workstation,  and  took  10.3  sec  to  complete.  A 
simulation  of  an  existing,  (i.e.,  real)  QCP  is  now  given. 

2.4  Application 

The  numericzd  simulation  of  zin  existing  plant  will  now  be  described.  The  template  for  the  initial  simulation 
is  taken  from  a  factory  located  in  Texas  that  gave  considerable  eimount  of  production  and  quaUty  control 
information  to  this  project. 

First,  a  brief  discussion  of  the  Texas  plant  is  given.  Then,  there  is  a  discussion  of  how  SOL  is  used  to 
create  a  simulated  model  of  that  template  plant.  The  program  details  are  followed  by  a  discussion  of  the 
pcirzimeters  chosen  for  the  different  units.  The  parameters  are  chosen  so  that  the  simulated  model  behaves 
numericailly  close  to  the  real  workceU. 

At  this  point,  the  reliable  simulated  model  is  extrapolated  so  it  matches  average  performance.  Such 
an  average  is  obtained  by  combining  the  production  performance  of  other  two  existing  companies.  The 
numericed  model  obtained  by  the  previous  process  is  then  extrapolated  to  the  case  of  robots  and  machine 
vision-based  inspection  workceUs.  The  parameters  for  these  units  are  obtained  from  the  experimental 
set  up  (see  Section  Five)  and  from  commercizilly  available  robot  performance  tables.  Lastly,  there  -  a 
discussion  of  the  simulation  resrdts. 

2.4.1  Simulation  of  the  Texas  Plant 
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RETORT  AREA 


The  overall  floor  view  of  this  plant  is  shown  in  Figure  7.  Pouches  (items)  have  to  undertake  two  100% 
in-line  inspections.  One  inspection  takes  place  before  the  retorting  process  eind  one  after  retorting.  The  two 
inspection  zones  Me  outlined  in  the  figure.  Two  independent  simulated  models  are  created  for  the  the  two 
inspection  processes.  In  this  case  the  retort  process  is  lengthy  so  it  has  large  buffers  before  and  after  the 
process.  This  condition  decouples  it  from  the  other  two  processes.  For  this  reason  independent  simulations 
give  the  same  results  as  the  unified  simulation.  The  simulations  ccirried  out  for  the  two  processes  will  now 
be  given. 

2.4.2  Pre-Retort  Workcell  Model 

Figure  8  shows  a  single  pre-retort  inspection  line  and  its  corresponding  simtilation  model.  The  simulation 
model  is  just  a  one  to  one  correspondent  of  the  real  inspection  line  using  the  basic  units  described  in  2.2. 
The  filler  is  replaced  by  a  source  tmit  emitting  pouches  at  the  same  rate.  Pouches  are  transported  by  the 
conveyor  belt  up  to  the  two  inspectors.  After  inspection,  inspectors  return  all  pouches  to  the  conveyor 
belt;  however,  in  the  rejil  plant,  discarded  pouches  are  put  on  the  conveyor  with  a  90°  rotation  with  respect 
to  the  origin2Ll  orientation.  In  the  simulation  model  pouches  have  a  vMiable  where  inspector  miits  write 
the  qucility  control  test  results.  It  is  up  to  the  transfer  operators  (real  case)  or  units  (simulated  model)  to 
classify  them  into  two  post-inspection  buffers. 

The  complete  pre-retort  inspection  line  is  shown  in  Figure  9.  It  has  four  inspection  workcell  in  parallel 
with  a  total  of  42  units  distributed  as  follows: 

•  4  sources. 

•  8  inspectors,  two  per  line 

•  4  conveyers  before  the  couple  of  inspectors,  and  four  conveyers  after. 

•  4  transfers.  Moving  and  selecting  pouches  after  inspection. 

•  8  buffers.  Two  per  line,  one  for  accepted  and  one  for  rejected  items. 
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•  8  transfers.  Two  per  line  moving  the  pouches  to  their  final  destination,  the  accepted  and  the  disposed 
units. 

•  1  discarded  unit. 

•  1  accepted  xinit.  This  one  replaces  the  real  retort  process  aind  the  reed  buffer  before  that  process. 

As  it  was  shown  in  the  previous  section,  the  implementation  of  a  simulator  for  this  plant  is  straight¬ 
forward  if  the  SOL  is  used.  The  implementation  is  just  a  matter  of  defining  the  plant  components  and 
describing  their  connectivity  by  meam  of  SOL’s  objects  {item,  source,  etc)  and  fimctions  {install,  connect, 
etc.)  However,  to  match  the  performance,  the  parameters  of  the  simulated  model  have  to  be  fine  timed. 
The  code  for  this  simulation  is  presented  in  Appendix  A. 

Figure  10  shows  the  performance  of  the  template  plant  for  a  28  days  production  run.  The  curve 
represents  the  total  number  of  pouches  being  inspected  each  day.  The  average  daily  production  is  100,000 
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pouches.  Figure  11  shows  the  total  number  of  defective  pouches  in  the  same  interval  of  time.  The  average 
number  of  discarded  pouches  is  500,  so  the  percentage  of  defective  pouches  is  then  around  0.5%.  In  aU 
the  data  gathered,  no  specification  was  made  about  the  independent  type  of  defects.  That  is  why  the 
nmnerical  simulation  implemented  here  uses  only  one  defect  with  TFR  =  0.005. 

In  order  to  obtain  similar  results  with  the  simulation  model  the  following  parzuneters  are  chosen: 

•  Source:  Rate  =1.5  pouches/sec 

•  Inspector:  Rate=0.347  pouches/sec,  inspection  time  2.9  sec. 

•  Conveyor:  Capacity  10  pouches,  trzinsport  time  10  sec. 

•  Buffer:  Capacity  100  pouches  for  the  one  before  the  accepted  unit,  eind  15  pouches  for  the  one  before 
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the  discarded  unit. 


•  Treinsfer:  Capacity  1  pouch,  transport  time  1.5  pouches/sec. 

It  is  t  here  that  »  iic  Texas  plant  is  one  of  the  most  efficien*  in  the  coimtry.  The 

foUi  ^  is  the  performance  of  other  two  compcuues: 

•  Company  II:  Four  pre-retort  ;s,  about  80,000  pouches  per  day.  10%  defect  rate. 

•  Company  iil;  52,000  pouches  per  day.  (defect  rate  unknown) 

Unfortimately,  detciiled  information  about  the  qucdity  control  components  and  the  v  kcell  coimectivity 
of  these  companies  was  not  available.  Hence,  the  general  layout  of  the  Texas  plant  will  be  used  together 
with  the  average  performance  of  all  three  companies  in  order  to  have  a  representative  generic  simulation 
model.  The  average  to  be  matched  is  then  around  70,000  pouches  per  day  with  a  discturded  rate  of  5%. 
This  means  that  out  of  the  70,000  pouches  3,500  are  rejected. 

The  simulation  of  the  pre-retort  workcell  matches  these  output  results  if  the  average  inspection  time 
is  set  to  3.9  seconds  (for  all  the  inspectors)  assuming  a  shift  of  8  hours  a  day  or  4.2  seconds  for  a  10  hour 
shift.  Nevertheless,  this  inspection  time  is  a  fimction  of  the  topology  (components  plus  coimectivity)  of 
the  plant.  As  it  was  mentioned,  the  Texas  plmts  was  used  as  a  prototype  for  these  matters. 

The  TFR  was  set  to  0.05  and  only  one  defect  was  used.  This  condition  miikes  sure  that  5%  of  the 
pouches  are  defective.  Each  one  of  the  four  source  units  was  set  up  to  generate  the  same  amoimt  of  pouches 
a  day  (1/4).  This  condition  bcilances  the  load  in  each  line  an  makes  the  emalysis  easier.  However,  for  more 
sophisticated  euicdysis,  it  is  possible  to  break  this  symmetry  condition  and  study  carefully  its  implications. 

2.4.3  Automatic  Pre-Retort  Workcell 

STPll  deals  mciinly  with  the  feasibility  of  automating  the  inspection  process.  That  is:  sources  conveyers 
and  some  transfer  units  may  change  a  little  in  the  automatic  version.  However,  inspectors  wiU  be  replaced 
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Figure  12:  Automatic  workceU  for  pre-retort. 
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completely.  For  this  reason  the  main  parameter  for  performance  compzurison  is  the  inspection  time  and  the 
inspection  accuracy. 

Inspection  accuracy  can  be  assumed  to  be  greater  or  equal  to  that  of  the  humein  inspector.  That  is 
because  as  it  will  be  shown,  the  inspection  time  can  be  compensated  by  increasing  the  number  of  inspectors 
working  in  parallel.  However,  the  accuracy  Ccin  not  be  increased  in  a  simple  way.  For  instance,  the  pouch 
can  be  double  checked,  but  this  will  double  the  implementation’s  cost  and  complexity  For  simulation 
purposes  it  is  assumed  that  the  accuracy  is  close  to  perfect  and  the  attention  is  focused  on  the  inspection 
time  (critical  for  the  plant’s  throughput). 

It  win  be  shown  in  Section  5  how  the  inspection  time  of  the  machine  vision  system  is  larger  than  the 
humm  inspection  time.  This  is  because  a  reliable  system  requires  a  time  consuming  image  processing. 
Although  the  inspection  time  for  the  machine  vision  obtained  in  preliminary  studies  (see  Section  5)  is  still 
relatively  high  (14  seconds),  the  inspection  time  used  for  the  workcell  simulated  here  is  assumed  to  be  8 
sec.  This  is  because  (as  it  will  be  discussed  in  the  conclusion  of  Section  5)  there  are  many  other  techniques 
(not  studied  presently)  that  will,  in  the  future,  lower  substantially  the  inspection  time. 

Figure  12  shows  m  2ilternative  pre-retort  workcell  based  on  automatic  inspectors.  The  workceU  layout  is 
quite  different  from  the  humein-based  workcell  but  the  parameters  for  sources,  conveyors  md  buffers  rem^n 
basically  the  same.  The  transfer  units  are  omitted  because  it  is  assumed  that  robotic  inspectors  with  several 
degrees  of  freedom  are  capable  of  performing  complex  transfer  movements.  The  intermediate  buffers  cire 
omitted  because  computer  controlled  units  can  optimize  the  synchronization  of  task  execution.  Another 
difference  is  that  the  inspection  process  is  Ccirried  out  for  16  hour  per  day.  With  these  characteristics,  this 
alternative  automatic  workcell  has  the  same  performance  as  the  generic  simulation  model. 

2.4.4  Post-Retort  Workcell  Model 

Figure  13  shows  the  inspection  line  and  the  respective  simulation  model  for  the  complete  post-retort 
inspection.  The  units  involved  in  this  model  are: 
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•  1  source,  replacing  the  retorting  process. 

•  1  post  retort  buffer. 

•  1  trans/er  between  the  large  post-retort  buffer  and  a  small  pre-inspection  buffer.  (15  pouches). 

•  1  pre  inspection  buffer  mentioned  above. 

•  1  transfer  \mit  capable  of  moving  5  pouches  at  the  same  time. 

•  1  conveyor  to  trcinsport  the  pouches  to  the  inspector. 

•  7  inspectors.  Inspecting  pouches  from  the  main  conveyor.  Accepted  pouches  are  put  back  on  the 
mtun  conveyor  while  discarded  pouches  go  to  a  secondary  conveyor. 

•  2  conveyors.  A  main  one  for  accepted  pouches  and  a  secondary  one  for  disctirded  pouches. 

•  1  process  (packaging).  Processing  accepted  pouches  from  the  main  conveyor. 

•  1  transfer  unit  that  ttikes  disctirded  pouches  from  the  secondary  conveyor  to  their  final  destination, 
the  discarded  unit. 

•  1  final  inspector. 

•  1  accepted  unit .  This  one  includes  the  boxing  process  that  has  nothing  to  do  with  the  overall  inspection 
process. 


The  post-retort  processes  may  look  quite  different  than  the  previously  anrilyzed  pre-retort  processes. 
Nevertheless,  from  the  simulation  point  of  view,  these  two  processes  are  quite  equivalent.  In  fact,  the  total 
number  of  inspectors  is  almost  the  stime  as  the  Texas  plant.  The  mtiin  difference  are  the  source  units. 
Pre-retort  processes  require  several  fillers  working  in  parallel  to  produce  the  right  amoimt  of  pouches.  The 
post-retort  source  is  just  a  large  buffer  where  the  pouches  coming  out  of  retorting  cure  being  stacked. 
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It  was  previously  mentioned  that  the  critical  components  of  the  QCP  are  the  inspector  imits.  The 
post-retort  cais  may  require  a  different  defect  detection  approach.  This  is  because  new  types  of  defects 
could  appear  rifter  the  heating  process.  For  the  simulation  presented  here,  this  matter  was  not  explored 
because  detailed  information  was  not  available  from  the  industry. 

The  automatic  workceU  model  proposed  for  post-retort  rs  the  same  as  the  one  for  pre-retort.  However, 
the  somce  imits  for  this  case  can  be  assumed  to  be  in  charge  of  the  air-drying  instead  of  the  pouch  filling 
eind  sealing  (see  Figure  i ).  It  should  not  make  a  difference  as  far  as  the  output  rate  is  conserved. 

The  code  for  the  post-retorting  simulation  is  listed  in  Appendix  A  for  future  reference. 

2.5  Conclusions 

The  quality  control  process  was  decomposed  into  several  simple  elements.  Numerical  models  for  each 
elements  were  easily  elaborated  beised  on  empirical  knowledge  and  common  sense.  The  abstract  models 
were  implemented  in  the  object  oriented  language  C+-h  generating  a  library  of  classes  (objects).  This 
partitioning  of  models  into  objects  represented  a  powerful  tool  for  numerical  simulation.  Simulation  for 
simple  and  complex  workcells  were  implemented  with  few  lines  of  code. 

A  simulator  was  built  to  resemble  both  numerically  and  topologically  an  existing  plant  from  which  a 
large  etmount  of  information  was  available.  The  simulator  was  extrapolated  to  match  the  average  perfor- 
mzince  of  other  two  plants.  A  fictitious  (non-existing)  robotized  pleint  was  proposed  to  perform  as  well  as 
the  average  template  plant.  Hence  it  was  proved  that  there  are  enough  controllable  parameters  to  design 
a  robot-based  plEUit  that  performs  as  well  or  better  that  a  human-based  plant.  However,  it  is  important 
to  observe  that  the  robot  based  inspector  must  at  least  match  the  error-rate  of  a  human  operator. 

Even  though  sufficient  data  was  not  available  to  create  a  detailed  simulation,  it  was  shown  that  numer¬ 
ical  simulation  is  a  powerful  tool  for  future  design.  The  numerical  model  can  be  extrapolated  to  extreme 
conditions  and  the  effects  on  individual  components  can  be  analyzed.  Such  flexibility  is  in  many  cases 
absent  in  read  prototypes. 
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3  Graphics  Simulation 

3.1  Introduction 

A  graphics  simulation  was  created  to  complement  the  numericeil  simulation,  called  the  Interactive  3D 
Graphical  Simulation  IGAS.  The  simulator  processes  are  distributed  among  two  computer  workstations. 
The  graphics  rendering  is  implemented  on  a  HP9000/360  graphics  workstation  with  graphics  accelerator, 
while  the  munerical  simulation  zind  the  user  interface  was  implemented  on  a  SUN  4/360  workstation.  The 
simulation  IGAS  was  developed  in  the  Human-Machine  Interface  Laboratory  where  several  tools  for  virtual 
reedity  simulation  are  available. 

IGAS  displays  animated  models  of  a  “human-based”  and  “robot-based”  inspection  workceU.  The  user’s 
interaction  with  the  3D  graphical  models  is  performed  by  means  of  a  track-ball  device  (Dimension6  Belli 
[15]),  which  allows  the  user  to  virtually  “fly-through”  the  3D  graphical  models. 

To  increase  the  simulation  performance,  the  munerical  simulation  and  the  Dimension  6  Ball  drivers  nm 
on  a  SUN/4  workstation  while  a  server-client  scheme  transfers  the  required  data  to  the  Hewlett  Packard 
workstation  via  an  ethemet  network. 

3.2  Graphics  Simulation 
3.2.1  General  Description 

Human-Based  Workcell  The  hiunan  based  station  is  shown  in  Figure  14.  The  block  diagram  of  such 
a  station  is  shown  in  Figure  15.  It  consists  of  two  inspection  stations,  one  before  and  one  after  a  generic 
process.  The  pouches  are  generated  in  a  sovuce  unit  aind  transported  by  the  first  conveyor.  Foiu  inspectors, 
two  at  each  side  of  the  first  conveyor,  visually  inspect  the  pouches.  The  pouches  accepted  by  the  inspectors 
are  placed  back  on  the  first  conveyor. 

The  rejected  pouches,  at  this  first  inspection  eirea,  are  placed  on  a  second  conveyor  which  is  perpendic¬ 
ular  to  the  first  one  and  goes  underneath  it.  This  conveyor  takes  all  the  defective  pouches  to  the  discarded 
unit  (not  shown  in  the  graphics  simtilation). 
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Figure  14;  Human  based  workcell;  Screen  picture 


PROCESS 


A  third  conveyor  transports  the  pouches  from  the  generic  process  to  the  second  inspection  station.  This 
inspection  area  has  also  fotir  inspectors  and  sh2u:es  the  same  conveyor  for  disczirded  pouches.  The  accepted 
pouches  will  remedn  on  the  third  conveyor  which  transports  them  to  an  accepted  unit. 

Robot-Based  Workcell  The  robotic  workceU  cem  be  considered  as  a  flexible-automation  implementa¬ 
tion  shown  in  Figure  16.  The  disposition  of  the  overall  workcell  consists  of  two  parallel  process  lines.  Each 
line  having  one  inspection  workcell  before  and  one  after  a  generic  process. 

Two  robot  manipulators  are  used  to  heindle  the  pouches.  In  this  particulcir  model  they  work  coupled 
to  turn  over  the  pouch.  Such  a  process  is  required  in  order  to  allow  the  vision  system  to  inspect  both  sides 
of  the  pouch.  The  vision  systems,  which  inspect  a  single  side  of  the  pouch,  are  located  before  and  after 
the  robot  cirms. 

The  conveyor  of  each  inspection  workcell  is  synchronized  with  the  inspection  process.  This  is  done  so 
that  the  pouch  remains  steady  (not  moving)  during  the  image  acquisition  process.  To  do  this,  the  conveyor 
stops  for  some  period  of  time.  The  robot  manipulators  zcre  also  synchronized  so  that  when  the  conveyor 
stops  there  is  eilso  a  pouch  present  vmdemeath  the  second  inspection  camera  system. 

3.3  System  Configuration 

The  overall  system  configuration  for  the  graphics  simulation  is  shown  in  Figure  17.  Two  dedicated  work¬ 
stations  are  used,  a  SUN  4/360  workstation  and  an  HP  9000/360  SRX  Graphics  workstation  with  a  98766 
Starbase  graphic  rccelerator.  The  HP  workstation  creates  the  3D  graphics  display,  while  the  SUN  work¬ 
station  rims  the  drivers,  the  Dimension  6  ball  controller  and  the  network  communication. 

Processes  on  the  same  workstation  communicate  via  shared  memory  segments  while  processes  on  differ¬ 
ent  workstations  communicate  over  the  ethernet.  For  the  latter,  a  server-client  network  scheme  is  employed. 
On  one  end  of  the  ethernet,  the  SUN_server  (running  on  the  SUN  workstation)  reads  the  serial  port  to 
transduce  the  user  commands  on  the  Dimension6  Track-ball,  then,  it  writes  the  appropriate  data  in  the 
network. 
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Figiire  17:  System  configuration  for  graphiccil  simulation. 
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The  Dimension6  Track-beill  controller  uses  optic  sensors  to  measure  forces  and  torques  applied  to  the 
ball  [15].  By  gently  squeezing  the  bedl  the  operator  applies  forces  and  torques  that  are  sent  over  an  RS-232 
serial  link  to  the  SUN.  The  Dimension  6  driver  uses  time  rate  control  to  map  force  and  torque  readings 
into  positions  and  orientations  of  3-D  objects. 

To  compensate  for  different  control  needs,  the  track-ball  keypad  has  been  used  to  change  the  force / torque 
gains  G.  In  this  way  by  using  a  high  gain  the  user  can  fly  faster  through  the  simulation.  A  small  gmn  G  is 
used  when  precision  positioning  (low  speed)  is  required.  Six  integers  (of  one  byte  each)  are  read  from  the 
track-bcdl  optic  sensors.  The  values  correspond  to  three  forces  Fx,Fy,Fz  and  three  torques  Mx,  M,,,  M^. 
These  values  are  multiplied  by  the  gain  factor  G  cind  then  sent  through  the  network  to  the  graphics 
application.  G  changes  from  0.2  to  1.0  in  steps  of  0.2  as  selected  by  the  user  using  the  track-ball’s  keypad. 

At  the  other  end  of  the  network,  the  HP -Client  (running  on  the  Hewlett  Packard  graphics  workstation) 
reads  the  ethemet  cuid  stores  the  received  data  in  the  corresponding  sh2ired  memory  segment.  This  infor¬ 
mation  is  then  used  by  the  graphics  routines  HP  JGAS  to  steer  the  view-point  (c2unera  point  of  view)  and 
simulate  a  real-time  flight-through  in  the  graphics  model.  In  the  graphics  code,  the  forces  GFx,  GFy,  GF^ 
are  used  to  displace  the  viewer  along  the  viewer  coordinate  system.  The  torques  GMx,  GMy,  GM^  are  used 
to  rotate  the  camera.  An  adtemative  simulation  mode  uses  the  same  data  not  to  move  the  camera  (which 
remains  fixed)  but  to  move  the  graplucs  model. 

3.4  Graphics  Simulation  Code 

To  improve  the  graphics  refresh  rate,  a  Display  List  technique  is  used  together  with  the  HP  Starbase 
graphics  library  [34].  A  special  data  structme  called  “segment-network”,  was  created  to  store  the  model 
of  the  virtual  scene.  Polygonal  objects  and  their  corresponding  4x4  homogeneous  transformations  [68,  34] 
Eire  placed  in  the  segment-network  in  a  hierarchicEil  way.  Labels  are  placed  in  the  segment-network  at  the 
position  of  transformations  of  the  objects  that  are  going  to  be  atumated. 

Animation  consists  of  editing  the  segment-network,  traversing  the  segment-network  to  update  the 
transformed  object  Eind  finally  displaying  the  resultauit  image.  Editing  the  segment-network  is  performed 
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by  changing  transformation  matrices  referenced  by  their  labels.  In  this  way  objects  can  be  rotated  and 
translated  dynamically. 

During  animation,  once  the  segment-network  h2is  been  edited,  the  history  of  the  whole  data  structure 
is  refreshed  traversing  the  hierarchiczdly  arranged  segment-network  from  the  higher-level  to  the  lower-level. 
Transformations  on  a  higher  level  are  applied  to  cdl  the  other  objects  in  the  lower  levels  which  belong 
to  the  same  branch.  In  this  way  it  is  possible  to  move  (translate  or  rotate)  a  complete  set  of  correlated 
objects  changing  their  parent  (root)  transformation  [68].  The  resulting  image  is  stored  in  a  secondary 
iin.ii^e  buffer.  Subsequently,  the  meiin  buffer,  holding  the  current  image  on  the  screen,  and  the  secondary 
buffer  are  swapped.  This  results  in  the  new  image  being  displayed. 

A  dedicated  data  base  was  created  which  contains  modeling  information  of  the  generic-objects:  color, 
vertex  list,  normal  to  the  surface  and  normal  to  the  vertex.  At  the  beginning  of  the  simulation,  the  data 
base  information  is  used  to  generate  the  objects  in  the  segment-network.  The  same  generic  objects  can  be 
used  several  times  to  build  similar  parts  in  the  segment-network.  Such  a  building-block  technique  saves 
storing  space  and  eases  the  global  off-line  changing  of  objects. 

Seeding  treinsformations  are  used  to  reshape  the  generic-object  to  its  final  form,  a  rotational  transfor¬ 
mation  reorients  child  objects  with  respect  to  their  parent  object,  and  finally,  a  translation  is  in  chetrge 
of  placing  the  elements  in  their  positions.  Figure  18  shows  a  block  diagram  of  the  graphics  simtilation 
progreim. 

The  average  refresh  rate  for  the  graphics  simulation  was  4  frames j  second.  If  the  viewer  is  relatively 
close  to  my  object  in  the  virtual  scene,  that  object  may  require  a  large  area  of  the  image  frame.  In  that 
case  lower  bmdwidths  of  1.2  frames  f  second  Eire  obtained.  When  the  viewer  is  placed  far  away  from 
the  graphics  model,  so  that  the  model  uses  a  fraction  of  the  screen,  the  graphics  refresh  rate  reaches  it 
maximum  of  10  frames  j  second. 

On  the  other  hmd  the  refresh  rate  is  also  a  function  of  the  complexity  of  the  segment-network  being 
displayed.  For  the  case  of  the  human-based  workcell  arotmd  600  polygons  were  used  while  for  the  robot- 
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Figtire  18:  Graphical  simulation  program:  Block  diagram 


based  workcell  only  300  polygons  were  required.  For  this  reason  the  refresh  rate  for  the  latter  was  30% 
faster.  The  low  bandwidth  obtained  in  this  simulation  was  a  consequence  of  the  graphics  workstation 
performance.  For  instance,  if  the  same  simulation  is  run  on  a  HP  755  workstation  it  is  possible  to  obtain 
50  or  more  frames! second.  Such  a  high  bandwidth  allows  to  implement  more  complex  2ind  reahstic  3D 
simulation. 

3.5  Conclusions 

Graphics  simulation  is  cdso  a  powerful  tool  for  complex  system  feasibility  studies.  In  this  Section  the 
qualitative  aspect  of  the  workcell’s  layout  were  easily  visualized  with  the  help  of  a  graphics  simulation 
(virtued  plant).  Interaction  with  a  3D  model  was  done  by  means  of  a  six  degrees  of  freedom  track-biill. 
Such  control  permitted  the  exploration  of  the  3D  graphical  model  by  simulating  a  user  “flight-through”. 

The  implemented  graphics  simulation  allowed  to  compute  the  workspace  requirements  for  human-based 
and  robot-based  workstations.  For  the  examples  studied  in  this  simulation,  the  space  requirements  for  robot 
workcell  system  (robot  manipulator  plus  machine  vision  systems)  are  larger  than  those  for  human-based 
workstation.  In  the  robot-based  layout  the  pouch  must  be  turned  aroimd  before  it  imdertakes  the  next 
vision  inspection  (in  order  to  inspect  both  sides).  This  procedure  is  implemented  by  two  coupled  robot 
arms  manipulators.  Considering  the  pay-load  requirements  (100g-500g)  such  a  device  can  be  implemented 
in  a  small  space.  However  the  two  vision  systems  may  require  a  larger  volume  which  can  exceed  a  human 
operator’s  work  space  volume. 

The  main  components  of  the  robot-based  workcell  were  graphically  modeled  in  this  Section.  The  robot 
manipulator  and  the  machine  vision  system.  Although  the  graphics  simulation  proved  to  be  a  very  efficient 
tool,  for  a  complete  layout  analysis  it  is  required  to  model  a  specific  design  of  the  robotic  workceU.  To 
do  that,  the  following  two  sections  will  aneilyze  the  two  elements  (robot-manipulator  and  machine- vision 
system)  in  detail.  This  wiU  explain  why  the  graphical  model  used  the  particular  3D  design  presented  in 
this  section. 
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Figure  19:  Pouch  manipulation  zones 


4  Workcell  Robot  Manipulators 

4.1  Introduction 

This  section  presents  the  description  of  two  robotic  workcells  which  can  be  used  for  pouch  manipulation. 
The  vision  system  is  described  in  the  next  section.  However,  the  pouch  manipxilation  needed  for  that 
system  will  be  introduced  here. 

The  workcell  area  is  decomposed  into  several  manipulation  zones.  The  particular  pouch  handling 
required  in  each  zone  is  discussed.  Based  on  the  overall  pouch  movement  requirements  and  the  manipulation 
zones,  two  robot  models  are  proposed.  For  each  case,  a  brief  analysis  is  made  in  terms  of  the  manipulator 
mechanism,  the  control  systems  required  and  their  performances. 

4.2  Pouch  Manipulation  Zones 

Three  main  zones  for  pouch  manipulation  cm  be  distinguished:  the  pre-inspection,  the  inspection  md  the 
post-inspection  zones.  In  the  pre-inspection  zone  the  pouches  mc  brought  from  sources  or  transport  systems 
to  the  inspection  zone.  In  the  inspection  zone  the  pouch  is  manipulated  so  that  the  vision  system  can  take 
all  the  required  images.  In  the  post-inspection  zone  the  pouches  cire  classified  as  accepted  or  rejected  and 
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returned  to  the  transport  systems  or  the  buffers.  Figure  19  shows  a  block  diagram  of  the  three  zones. 

What  follows  is  a  description  of  the  relevant  details  of  each  of  the  pouch  meinipuiation  zones.  Associated 
to  each  of  the  zones  is  a  machine  (a  robot  or  a  fixed- automation  device)  which  heindles  the  pouches.  Such  a 
machine  will  be  CciUed  a  manipulator  and  will  be  referenced  together  with  its  zone  name,  i.e.:  pre-inspection 
manipulator,  etc. 

4.2.1  The  Pre-Inspection  Zone 

In  many  cases,  pouches  are  brought  to  the  inspection  zone  via  conveyors.  In  those  cases  the  pouches 
usuzdly  come  in  random  orientations  eind  positions  (along  the  conveyor).  For  human  operators,  locating  and 
grasping  a  pouch  from  a  conveyor  requires  neither  tr^uning  nor  special  abilities.  In  contrast,  the  same  task 
could  represent  a  rezd  challenge  for  simple  robot  based  workcells.  For  this  reason,  some  of  the  automation 
models  proposed  in  this  research  require  that  the  incoming  pouch  be  located  and  oriented  between  certain 
ranges.  Otherwise,  extra  vision  systems  or  sensing  systems  will  be  required  so  that  the  robot  manipulators 
be  able  to  perform  precise  grasping.  Detziils  on  this  matter  are  addressed  when  the  particular  models  are 
described.  However,  it  is  important  to  point  out  that  if  any  extra  devices  are  required  in  order  to  locate 
or  orient  the  pouches,  such  devices  must  be  considered  part  of  the  pre- inspection  mzinipulator. 

4.2.2  The  Inspection  Zone 

Human  inspectors  usually  rotate  or  stretch  the  pouches  during  inspection  to  visualize  hidden  areas  or 
to  remove  wrinkles.  However,  it  has  been  established  that  the  Vision  System  (developed  by  the  STPll) 
requires  only  that  the  pouch  be  stretched  out  during  the  image  registration  process.  The  pouch  must  also 
be  fixed  and  held  steady  dming  this  process.  Both  sides  of  the  pouch  have  to  be  available  to  the  cameras 
for  a  two-side  inspection  process. 

The  two  side  problem  can  be  solved  in  three  ways: 

•  The  inspection  zone  manipulation  is  designed  so  that  a  double  Vision  System  can  take  images  of 
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Figure  20:  Pouch  mainipulation  in  the  inspection  zone 

both  sides  at  the  same  time. 

•  The  inspection  zone  m2inipulation  is  such  that  the  pouch  is  rotated  180  degrees  allowing  a  single 
Vision  System  to  “see”  both  sides  of  the  pouch. 

•  The  inspection  zone  manipulation  has  two  consecutive  Vision  Systems  and  the  pouches  are  turned 
over  before  they  go  through  the  second  Vision  System. 

Figure  20  shows  schematically  the  three  possible  manipulation  methods  for  the  inspection  zone.  For 
futme  reference,  they  will  be  called  inspection  zone  manipulation  types  A,B  and  C. 
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4.2.3  The  Post-Inspection  Zone 

This  part  of  the  pouch  manipulation  does  not  represent  a  major  challenge.  It  consists  basically  of  removing 
the  pouches  from  the  inspection  zone  and  placing  them  into  conveyors  or  containers.  Two  possible  desti¬ 
nations  have  to  be  eiUowed  in  this  zone.  The  accepted  and  the  rejected  pouches  should  take  different  paths. 
However,  no  special  time  should  be  spent  on  how  the  pouches  are  stored  or  stacked  cifter  this  process.  The 
main  objective  of  the  post-inspection  manipulator  is  then  to  free  the  inspection  manipulator  from  already 
inspected  pouches. 

Some  manipulation  models  do  not  allow  a  clecur  distinction  between  zones.  This  happens  when  the  same 
manipulator  is  used  to  perform  pouch  hiindling  for  two  or  three  of  the  zones  described  above.  For  example; 
a  robot  eirm  used  to  pick  incoming  pouches  from  the  conveyor,  locate  the  pouches  tmder  the  Vision  System 
performing  the  stretching  and  rotation  required  for  inspection,  and  then  dropping  the  pouch  back  on  the 
outgoing  conveyor. 

4.3  Types  of  Automation 

The  different  levels  of  automation  affect  drastically  the  implementation.  When  automating  a  meinual 
process  there  Eue  two  types  of  approaches  that  need  to  be  considered:  flexible- automation  and  fix^d- 
automation. 

To  use  a  single  multiple- degree  of  freedom  robot  arm  to  handle  the  pouch  in  all  the  manipulation 
zones  is  a  typical  case  of  a  high-level  flexible- automated  workcell.  In  this  case  the  manipulation  zones 
represent  just  parts  of  the  robot  work  envelope  and  not  separate  pieces  of  hardware.  It  is  considered  to  be 
a  high-level  automation  because  the  manipulator  is  a  multi-degree  of  freedom  arm  with  complex  task  and 
trajectory  planning  capabilities  (given  by  its  mechanicrd  design  and  its  control  interface).  It  is  considered 
a  flexible-automation  because  most  of  the  high-level  systems  are  very  flexible  cind  programmable  in  terms 
of  tasks  and  changing  environments.  The  same  robot  can  be  adapted  for  different  pouch  sizes  or  different 
inspections  requirements.  It  cein  be  used  in  other  places  in  the  plant  to  do  packing  or  p2dletizing. 
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In  contrast  to  high-level  flexible-automation  is  fixed-automation.  In  this  case  specialized  hardware 
devices  are  devoted  100%  to  perform  a  particular  part  of  the  whole  process.  This  low-level  automated  unit 
has  few  degrees  of  freedom  ^lnd  no  complex  controller  is  required.  For  exeimple  a  rejected  pouch  can  be 
pushed  off  the  conveyor  by  a  simple  pneumatic  cylinder  activated  by  a  single  digitad  signal.  A  vibratory 
conveyor  can  be  used  to  align  the  pouches.  The  stretching  of  the  pouch  can  be  performed  by  a  dedicated 
device  built  for  the  specific  pouch  size  inspected. 

The  fixed-automation  devices  eire  always  useful  to  supplement  flexible-automation  devices  present  in 
the  workcell.  Such  devices  can,  in  many  cases,  release  the  generic  robots  (flexible-automated)  from  doing 
easy  tasks,  improving  significantly  the  overall  process  time.  Applications  can  be  optimized  in  terms  of  cost 
and  performance  if  an  adequate  balcince  between  soft  and  hard  automation  is  used  when  designing  md 
implementing  the  system. 

4.4  Hardware  Models 

The  two  hardweire  models  are  now  given  The  purpose  is  not  to  present  the  models  up  to  the  point  of 
prototjrpe  design.  The  idea  is  to  outline  the  (possible)  components  needed  to  build  a  workcell  in  terms  of 
robotic  requirements.  This  is  why  this  section  does  not  include  detailed  designs.  While  implementation  of 
any  of  the  models  discussed  here  is  not  finsdized,  all  the  parEimeters  are  kept  between  reasonable  rmges. 
Also,  a  discussion  on  the  possible  driver  devices  required  to  control  the  robot  workcell  is  included. 

A  schematic  representation  of  the  vision  system  is  outlined  in  Figure  21  for  the  case  of  a  single  side  in¬ 
spection.  In  general,  the  vision  system  should  not  interfere  directly  with  the  pouch  memipulation,  although 
my  meinipulator  design  must  satisfy  aU  the  requirements  demanded  by  the  vision  system. 

As  it  will  be  shown  in  the  next  section,  the  vision  system  and  the  image  processing  are  the  most  critical 
parts  of  the  STPll.  The  vision  system  controller  is  also  described  in  Section  5.  This  prirticular  device  is 
considered  independently  of  the  mrinipulator  controllers  to  facilitate  the  malyses.  However,  it  is  clem  that 
certain  signrds  must  couple  the  two  systems  to  allow  a  synchronized  operation. 

The  two  conceptual  models  designed  specifically  to  deal  with  the  pouch  manipulation  required  for 
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STPll.  are  now  presented.  The  models  make  used  of  the  concepts  of  flexible  and  hard  automation.  It  will 
be  clear  how  the  performance  and  implementation  are  related  with  the  type  of  automation. 

4.4.1  Model  I 

General  Description  This  is  a  more  general  case  of  the  example  cited  in  4.3.  Here  two  robot  ma- 
nipxilators  are  used  in  the  pre- inspection  and  the  post-  inspection  zones.  The  inspection  manipulator 
is  a  flxed-automation  device.  The  model  is  presented  in  Figure  22  cind  described  in  terms  of  the  three 
mzaiipulation  zones  as  follows: 

•  Pre-inspection  manipulator  The  first  robot  grasps  the  pouch  from  a  feeder  conveyor  and  transfers  it 
to  the  inspection  meinipulator.  This  robot  is  a  four  degrees  of  freedom  manipulator.  Three  degrees 
cire  used  to  position  the  end  effector  on  top  of  the  pouch,  the  fourth  degree  is  used  to  compensate 
the  possible  misalignment  of  the  incoming  pouch.  The  pouches  are  grasped  by  means  of  a  vacuiun 
gripper  end-effector. 
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•  Inspection  Manipulator  A  special  fixed- automation  device  is  used  here.  It  is  ceilled  a  stretching  table 
and  is  shown  in  Figure  23.  It  consists  of  four  retractile  rods  — prismatic  joints —  each  one  having  a 
pneumatic  gripper  at  its  end.  The  process  of  stretching  the  pouch  is  as  follows: 

-  Step  1:  The  pouch  is  brought  by  the  pre-inspection  manipulator  to  the  stretching  position  in 
the  middle  of  the  four  retractile  rods. 

-  Step  2-.  The  retractile  rods  approach  the  pouch  at  the  corners  until  the  proximity  sensor  installed 
inside  the  gripper  detects  the  pouch.  At  this  moment  the  four  grippers  are  closed  holding  steady 
the  pouches  by  the  seeded  eireas  at  the  comers.  The  pre- inspection  manipulator  releases  the  pouch 
and  moves  away. 

—  Step  3:  The  rods  retract  stretching  out  the  pouch.  At  this  moment  the  images  are  registered  in 
both  Vision  Systems  which  take  simultaneous  images  of  both  faces  of  the  pouch. 

-  Step  4-  The  post-inspection  manipulator  comes  into  the  inspection  zone  and  holds  the  pouch 
with  a  vacuum  gripper  while  the  stretching  table  opens  the  grippers  releasing  the  pouch. 

•  Post-inspection  M  anipulator  This  is  a  3  degrees  of  freedom  manipulator  that  takes  the  pouch  from  the 
stretching  table  (step  4  of  stretching  process)  and  puts  it  into  outgoing  conveyors.  The  orientation 
of  the  pouch  is  not  relevant  in  this  mcinipulation  zone.  Two  conveyors  are  available,  one  for  accepted 
and  one  for  rejected  pouches.  For  this  robot  to  be  able  to  decide  where  to  place  the  pouch  the 
machine  vision  system  must  have  an  evcduation  on  the  pouch’s  quality  before  the  robot  finishes  its 
trajectory.  Otherwise  the  robot  will  dump  the  pouches  in  a  single  conveyor  cind  another  device 
(fixed-automation)  removes  the  defective  pouches  later.  This  latter  procedure  is  attempted  to  avoid 
keeping  the  post-inspection  manipulator  in  a  hold. 

The  stretching  table  (ST)  is  a  typical  example  of  fixed-automation.  Such  a  device  must  be  designed 
and  implemented  for  the  specific  needs  of  the  workceU.  Even  though  there  are  meiny  ways  to  design  the 
ST,  only  one  is  presented  here. 
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The  retractile  rods  can  be  driven  by  pneumatic  cylinders.  Figure  24  shows  a  schematic  outline  of  the 
proposed  device.  A  single  piston  A  pushes  the  rod  R  forward  towards  the  pouch  .  A  spring  S  is  used  to 
retract  the  rod  once  the  pneumatic  piston  A  has  been  decompress.  An  extra  pneumatic  cylinder  B  is  used 
to  quickly  stop  the  rod’s  forwcird  motion.  The  rod  is  stopped  once  the  pouch  has  been  detected  by  the 
gripper  sensor  D.  Break  B  is  used  because  of  air  compressibility  which  makes  it  very  difficult  to  control 
the  position  of  pneumatic  actuators  A.  Servo  positioner  devices  (e.g.  DC  serve  motors)  that  can  perform 
such  a  precise  positioning  control  Me  more  complex  and  expensive. 

The  sensor  (or  sensors)  D  located  at  the  gripper  detects  the  corner  of  the  pouch  and  can  be  implemented 
by  an  infrMed  or  ultrasonic  proximity  detector.  The  function  of  this  sensor  is  to  determine  when  the 
retractile  rod  should  stop  so  that  the  grasping  is  safe.  Without  these  sensors  the  gripper  could  miss  the 
comer  or  could  destroy  the  pouch. 

If  the  conditions  Me  such  that  the  pouch  is  edway  placed  in  the  same  exact  position,  the  stretching 
table  can  be  simplified.  This  can  happen  if  the  position  and  orientation  of  the  pouch  at  the  pre-inspection 
zone  Me  completely  determined.  In  this  case,  the  pre-inspection  manipulator  can  place  the  pouch  precisely 
in  the  middle  of  the  stretching  table  eiligned  with  it.  For  this  condition  the  stretching  table  does  not  need 
either  proximity  sensors  or  breaking  systems. 

The  gripper  actuator  is  a  double  action  pneumatic  actuator  C.  The  pMticulM  design  shown  in  Figure 
24  uses  a  double-side  sliding  gear  mechanism.  As  the  cylinder  road  slides,  it  drags  with  it  two  geMed 
wheels  attached  to  each  finger.  A  double  action  cylinder  has  been  chosen  to  gUMantee  the  complete  release 
of  the  pouch  before  the  retractile  rod  retracts.  If  for  «iny  reason,  one  of  the  four  grippers  is  stiU  holding 
the  pouch  while  the  rods  Me  retracted,  the  pouch  can  be  torn  or  pulled  out  of  the  vacuum  grippers. 

Implementation  and  Control  The  pMticulM  elements  that  can  be  used  for  implementation  Me  now 
discussed.  The  suggested  items  Me  not  necessMily  the  best,  they  are  just  examples  of  specific  available 
products  suitable  to  perform  a  required  task,  or  reqmred  to  build  a  specific  fixed-automation  device.  Also, 
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Figure  24:  The  retractile  rod  of  the  stretching  table 
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a  brief  discussion  about  the  electronic  devices  (interface,  computer,  etc)  required  to  control  the  robot 
workcell  is  included.  If  possible,  an  outline  of  the  overall  system  architecture  is  presented. 

Many  commercially  available  robot  manipulators  can  be  used  to  implement  the  pre-inspection  and 
post-inspection  robots.  For  excunple  the  popvd£ur  industri2il  IBM  7535  Sceira  is  adequate.  Four  reasons 
support  this  choice: 

•  The  required  work  envelop  is  small. 

•  The  required  speed  is  high. 

•  The  arm  geometry  is  adequate  for  the  task. 

•  The  required  payload  is  very  low. 

However,  many  other  robots  similzur  to  the  Scara  can  be  used.  For  instance,  the  AR-H250  made  by 
Hirata  Industrial  Machineries  Co.  has  the  Scara  geometry  but  a  lower  payload  and  also  a  lower  cost.  The 
retail  price  of  one  of  these  units  including  their  controllers  are  between  $20,000  and  $40,000. 

A  commercial  industrial  robot  of  this  type  comes  with  its  own  controller.  This  fact  eases  their  applica¬ 
tion  because  such  devices  are  already  loaded  with  the  programs  needed  to  optimally  utilize  the  manipulator. 
Those  controllers  often  include  a  high-level  language  interpreter  for  easy  programming. 

One  of  the  elements  that,  in  genereil,  is  customized  in  any  industrial  robot  is  the  end-effector  (such  a 
device  is  task-dependent).  For  the  case  of  MRE  pouches,  it  is  advisable  to  use  a  vacuum  gripper.  The 
reason  is  that  the  very  soft  materiail  of  the  pouch  easily  squeezes  imder  low  pressure.  For  this  reason  its 
exact  shape  is  not  weU  defined  making  it  difficult  for  a  finger-like  end-effector  to  grasp  with  precision  md 
stability.  On  the  other  hmd,  the  pouch  little  weight  (from  lOOg  to  500g)  allow  the  use  of  low  vacuum 
levels  for  a  safe  grasping.  The  latter  also  guarcintees  that  suction  does  not  damage  the  pouch. 

Mziny  commercial  vacuum  grippers  are  available.  For  instance,  the  VG102-1  made  by  SENSOFLES/ASTEK 
is  a  good  example  of  one  such  device.  The  price  of  this  units  is  relatively  low  (aroimd  $500).  This  type  of 
device  includes  its  own  vacuiun  pump  which  m^lkes  implementation  a  matter  of  instaiUation.  A  wide  range 
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of  levels  of  vacuum  can  be  chosen  cind  the  activation  or  inactivation  of  the  vacuum  gripper  is  controlled  by 
a  digital  signal. 

In  general,  the  end-effectors  do  not  require  an  extra  controller.  They  are,  in  many  cases,  controlled  by 
a  few  signeds.  Industried  robot  controllers  have  severed  general  purpose  ports  which  can  be  progrcimmed 
within  meinipulator  program.  In  this  way  the  gen»  ric  end-effector  becomes  another  extra  degree  of  freedom, 
making  it  possible  to  include  its  behavior  into  the  overall  control  strategy. 

Once  the  robot  2ind  its  controller  have  been  installed,  all  that  is  needed  is  to  program  the  required 
movement  sequences  using  the  high-level  language  supplied  by  the  controller.  A  way  to  interface  with 
the  controller  is  via  a  host  computer.  All  robot  controllers  have  a  host  computer  interface  that  allows 
communication  with  a  general  propose  computer.  This  facilitates  the  simultaneous  use  of  several  robots 
in  a  single  workceU.  A  host  computer  is  used  to  monitor  and  command  all  the  robots. 

The  host  computer  does  not  directly  drive  actuators  as,  shown  in  Figure  25.  The  host  computer 
monitors  the  task  execution  programmed  in  each  robot  and  it  is  able  to  initiate,  change,  stop  or  resiune 
the  activities  of  each  one  of  them.  A  host  computer  ideally  should  interface  with  aU  the  devices  present 
in  the  workcell  so  that  it  cein  synchronize  the  activities.  The  host  computer  also  interfaces  with  a  human 
operator  who  has  the  ultimate  control  over  the  entire  process. 

Some  of  the  devices  coimected  to  the  host  computer  are  external  sensors.  Sensors  are  placed  in  criticed 
locations  to  directly  monitor  the  state  of  the  workceU.  The  information  from  the  sensors  aUows  the  host  to 
take  action  in  real-time  according  to  the  workceU  state.  The  feedback  signals  from  the  sensors  are  required 
in  order  to  build  an  accurate  and  reUable  workceU.  Figure  26  shows  a  schematic  representation  of  the 
feedback  loop  for  the  case  of  n  robots. 

Sensor  information  cein  also  be  sent  to  the  robot  controUers.  That  is  the  case  of  sensors  that  directly 
affect  an  specific  robot.  At  the  robot  controUer,  such  information  can  be  used  to  take  decision  without 
consulting  the  host.  For  instcince,  a  coUision- detection  device  must  commumcate  directly  with  the  controUer 
of  the  robot  involved. 
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Figure  26:  Control  loop  for  robot  workcell 


On  the  other  hand,  since  the  I/O  port  of  the  host  can  be  limited,  the  robot  controller  can  be  used  as  a 
general  interface.  That  is,  by  encoding  and  sending  data  (together  with  robot  data)  through  the  controller 
communication  port.  Such  a  scheme  reduces  the  n\xmber  of  cables  and  interfaces  because  the  usual  links 
between  host-controllers  are  serial  ports. 

4.4.2  Model  II 

General  Description  The  following  model  is  based  mostly  on  fixed-automation.  However  a  robot 
manipulator  described  here  may  be  used  in  other  applications  as  well.  It  is  considered  a  fixed  omated 
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device  because  it  has  to  be  built  for  this  specific  task,  and  is  not  commercjally  available. 

Model  n  uses  a  custom  designed  ceirtesiem  robot  and  a  stretching  table  (described  in  4.4.1)  as  shown  in 
Figure  27.  The  robot  has  two  simple  3irms,  the  first  one  A  is  the  pre-inspection  manipulator  2ind  second  one 
B  is  the  post-inspection  manipulator.  Each  arm  is  equipped  with  a  vacuum  gripper  end-effector.  Because 
of  the  inspection  cycle,  the  two  arms  may  be  driven  by  the  same  actuator.  However  as  it  will  be  shown, 
larger  flexibility  is  reached  if  they  eire  made  independent. 

The  arms  move  eJong  a  common  horizontal  rail  above  the  conveyor.  Even  though  the  reiil  is  pareillel 
with  the  conveyor,  it  is  shifted  to  the  side  so  that  it  does  not  interfere  with  the  vision  system. 

The  cycle  to  inspect  a  single  pouch  is  as  follows: 

•  While  turn  A  is  grasping  a  new  pouch  (from  the  feeder  conveyor)  to  be  inspected,  arm  B  is  removing 
an  inspected  pouch  from  the  stretching  table. 

•  Then  arm  A  moves  so  that  its  end-effector  is  on  top  of  the  stretching  table  tind  arm  B  moves  so  that 
it  is  on  top  of  the  outgoing  conveyor  (towards  the  right  of  the  figure). 


•  Arm  A  deposits  the  new  pouch  on  the  stretching  table  while  arm  B  releases  the  inspected  pouch  in 
the  outgoing  conveyors.  If  the  inspected  pouch  must  be  disceurded,  Eum  B  releases  the  pouch  on  top 
of  the  rejected  conveyor  before  it  reaches  the  accepted  conveyor  (see  Figure  27). 

•  Arm  A  moves  away  from  the  stretching  table  (towards  the  left  of  the  figure).  The  vision  system 
takes  the  required  images. 

•  Arm  B  moves  back  so  that  its  gripper  is  on  top  of  the  stretching  table.  If  the  inspection  process 
requires  more  time,  then  arm  B  stops  so  that  it  does  not  interfere  with  the  vision  system. 

Components  and  Implementation  The  stretching  table  used  in  Model  II  is  the  same  as  the  or‘^  used 
in  Model  I. 
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Arm  A  hcis  to  move  up  and  down  to  pick  the  pouch  from  the  conveyor  without  interfering  with  the 
pouch’s  horizontal  displacement  (tracking).  On  the  other  hemd,  arm  B  must  move  up  and  down  to  remove 
the  pouch  from  the  stretching  table  to  avoid  colliding  with  the  pouch  laterally.  This  horizontal  movement 
of  the  arms  Ccin  take  place  simultaneously  and  does  not  require  any  precise  displacement  control.  Hence  it 
can  be  implemented  with  pnemaatic  cylinders.  The  horizontal  movement  tdong  the  rail  ccin  be  implemented 
by  metms  of  an  AC  or  DC  servo  motor. 

The  position  of  the  pouch  (on  the  feeder  conveyor)  c  n  be  detected  with  a  mechanical  micro-switch  or 
an  infra-red  light  betim  sensor(very  low  cost  methods).  Assuming  that  the  pouches  are  always  aligned  on 
the  feeder  conveyor,  arm  A  does  not  need  additicntd  degrees  of  freedom  to  compensate  for  deviations  (i.e. 
rotation  about  the  perpendiculzu:  axis  to  compensate  Ihe  pouch  orientation  or  treinslation  to  compensate 
for  displaced  positions  on  the  conveyor).  In  this  case,  the  mentioned  sensors  are  enough  to  guarcintee  that 
arm  A  will  descend  in  the  middle  of  the  incoming  pouch.  As  it  was  said  earlier,  this  can  simplify  the 
stretching  table  design  as  well. 

It  may  be  possible  to  synchronize  the  whole  process  so  that  both  arms  can  be  displaced  simultcineously 
with  the  scime  actuator.  However,  such  a  coupled  system  makes  it  difficxilt  to  optimize  the  control  and  to 
compensate  for  random  errors,  e.g.  arm  A  looses  its  pouch  and  it  has  to  go  back  to  feeder  conveyor  while 
arm  B  should  continue  its  normal  activity. 

Some  of  the  possible  actuators  that  can  be  used  to  implement  the  movement  are: 

•  A  DC  or  AC  motors  capable  of  moving  both  grippers  at  reasonable  speed  (2  feet/sec).  The  price  of 
this  unit  varies  through  a  wide  range,  from  $100  to  $1000.  A  reasonable  motor  is  a  DC  servo-motor 
TTB2-2933-3020-HA  made  by  INLAND  Inc.  It  has  1.5  HP,  3.4Lb.-Ft  torque,  brake  and  motor-tach. 
The  cost  of  this  unit  is  $400.  It  requires  a  servo  amplifier  that  costs  $800. 

•  Two  pneumatic  cylinders  to  move  the  gripper  up  and  down.  A  single  action  cylinder  ESN- 1-2  made 
by  FESTO  Inc.  has  lin  bore  and  2in  stroke  and  costs  $70. 
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•  The  vacuum  grippers  cire  the  same  ones  used  in  Model  I. 

As  all  the  automatic  systems  described  here.  Model  II  requires  a  computer  controller.  For  custom 
designed  robots  there  are  generic  controllers.  Such  devices  are  microcomputers  based  on  standcird  micro¬ 
controller  or  micro-processor  chips.  They  can  be  accessed  by  a  host  computer  or  they  may  have  their  own 
console.  In  either  case,  programs  must  be  written  to  generate  a  library  of  routines  specialized  in  controlling 
the  particular  manipulator. 

A  total  cost  of  Model  11  is  around  $2000  for  the  complete  unit.  It  is  important  to  note  that  the  ciir 
compressor  and  the  computer  controller  can  be  shared  among  several  of  these  units,  reducing  the  costs. 
However,  the  detailed  design  and  its  prototyping  (for  testing)  add  extra  costs,  amounts  that  are  beyond 
the  scope  of  analyses  of  this  research. 

4.5  Conclusions 

The  price  of  an  automatic  workcell  is  a  function  of  the  automation  level.  If  fixed-automation  units  me  used, 
the  price  is  low  (in  the  range  of  $10,000  to  $30,000).  However,  the  reliability  may  be  low  too  compared 
to  a  higher  automation  level  system.  Such  systems  me  implemented  by  means  of  flexible-automated  units 
like  robot  mm  m2uupulators.  Systems  based  on  flexible-automation  are  expensive  (in  the  range  of  $30,000 
to  $80,000  or  more!).  Nevertheless,  these  systems  me  very  reliable  and  flexible. 

Custom  designed  systems  can  be  used.  These  systems  me  a  mixture  of  flexible  and  fixed- automation 
(for  instance,  special  designed  arm  manipulators).  For  these  systems  the  overall  cost  decreases  but  the 
designing  eind  prototyping  of  the  custom  designed  robots  adds  costs  whose  analyses  me  not  considered  in 
this  report. 

Making  use  of  currently  used  machinery  cein  reduce  the  cost  and  complexity  of  the  workceU  imple¬ 
mentation.  As  mentioned  in  this  section,  the  precise  grasping  of  a  random  oriented  and  positioned  pouch 
increases  the  degree  of  complexity  of  the  robotic  system;  increasing  in  this  way  the  implementation  cost. 
But  for  instance,  if  an  inspection  workcell  is  coupled  directly  to  the  current  filler  machine,  there  is  no  need 
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for  grasping  pouches  randomly  from  a  conveyor.  These  methods  mEike  use  of  whatever  mechanisms  the 
filler  machines  use  to  hold  steady  the  empty  pouch,  fill  it  cind  seal  it.  This  particular  solution  requires 
further  ainalysis  of  current  used  machinery.  Such  analysis  is  suggested  as  future  work. 
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5  The  Vision  System 

5.1  Introduction 


This  Section  presents  a  discussion  of  the  initial  vision  system  proposed  by  the  STPll  research  team.  It 
contains  also  a  discussion  of  the  proposed  modifications.  This  Section  describes  the  first  vision  inspection 
prototype.  Based  on  this  experience  some  possible  impiu.cinents  of  the  system  and  some  ^lltemative 
solutions  will  be  presented  in  the  final  part  of  this  section. 

The  vision  system  is  the  most  critical  pcirt  of  the  quality  control  of  MRE  pouches.  100%  of  the  inline 
inspection  is  visual.  In  current  manuzd  plants,  human  operators  look  at  the  pouch  for  several  seconds  md 
based  on  their  experience  (and  training)  determine  whether  or  not  a  given  pouch  is  a  defective  one.  The 
mission  of  machine  vision  is  to  develop  a  system  capable  of  replacing  the  human  inspector  in  this  task. 
Such  a  system  must  be  as  reliable  as  the  human  operator  and  must  be  able  to  perform  the  inspection  task 
in  a  reasonable  time. 

The  objective  is  not  to  develop  a  system  that  can  perform  the  inspection  in  less  time  than  a  human 
inspector  can.  It  has  been  shown  in  Section  1  how  the  overall  plant  topology  can  be  designed  to  compensate 
for  a  slower  robot  based  workcell.  However  the  reliability  of  the  machine  vision  is  very  importemt.  Here 
reliability  means  the  success  rate  of  the  system,  in  other  words,  the  percentage  of  time  that  a  machine 
vision-based  inspector  is  capable  of  successfully  determining  the  quedity  of  the  pouch. 

The  performance  of  the  machine  vision  inspector  affects  in  such  a  way  the  overedl  plant  production 
quality,  that  it  czin  not  be  compensated  with  simple  topological  or  parametric  cheinges.  The  MRE  defects 
were  classified  into  four  types,  each  of  them  with  several  classes  as  follows: 

Type  1  -  Critical 

a)  swollen  pouch 

b)  tear,  cut,  hole,  or  abrasion/silver 

c)  fold  over  wrinkle 
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d)  entrapped  matter  in  seal 

e)  delamination 

f)  leakage(open  seed,  short  seal) 

Type  2  -  Major  A 

a)  uncle^m  pouch 

b)  missing  or  incorrect  labeling 

c)  impression  of  seed  area 

d)  less  than  3/16”  tear  notch  seal  area 

Type  3  -  Major  B 

a)  color  change  after  thermal  processing 

b)  incorrect  heat  seal  dimensions  (minimum  1/16”  seal  width) 

c)  dislocated  closure  seal  (<  1”  from  top  of  pouch  to  bottom  of  seal) 

d)  incorrect  pouch  external  dimensions  (4.75”x8.125”) 
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Type  4  -  Minor 

a)  missing  or  misshapen  tear  notch 

b)  dislocated  tecir  notch 

c)  incorrect  tear  notch  depth 

The  machine  vision  technique  discussed  in  this  section  deals  only  with  the  defects  that  expose  the 
pouch’s  metal  coating,  such  as  tear,  cut,  hole,  abrasion,  delamination.  As  it  will  be  clear  later,  some  types 
of  imclean  or  swoUen  pouches  are  also  detectable  by  the  vision  technique  presented  here. 

5.2  Background  Theory 

The  detection  of  defects  by  machine  vision  techniques  impUes  the  analyses  of  images  taken  from  the  pouches. 
Such  images  cire  made  possible  because  pouches  (as  most  other  objects)  reflect  visible  light.  Reflection  and 
refraction  are,  indeed,  very  complicated  processes  whose  thorough  ancdysis  involves  the  knowledge  of  the 
electromagnetic  interaction  of  photons  and  matter.  Fortunately,  for  the  model  imder  consideration,  such 
ein3dyses  Me  not  required. 

The  behavior  of  the  phenomenon  involved  here  can  be  modeled  with  the  help  of  simple  empiriceil 
descriptions.  Meiny  of  such  descriptions  do  not  necessMily  correspond  to  a  real  physical  phenomenon,  but 
for  the  scope  of  this  study  they  are  appropriate. 

We  start  by  introducing  simple  definitions  on  which  the  defect  detection  model  is  based. 
Diflfuse-Reflection:  Also  c^lUed  Lambertian  reflection.  Dull,  matte  surfaces  like  chalk  exhibit  diffuse 
reflection.  Such  a  surface  appears  equally  bright  from  almost  all  view  points.  This  is  because  these  surfaces 
reflect  light  with  (almost)  the  same  intensity  in  all  directions.  The  reflected  intensity  is  a  ftmction  of  the 
angle  $  between  the  normal  to  the  surface  N  and  the  direction  of  the  incident  light  beam  Li  (See  Figure 
28a).  For  a  single  point-light -source  the  Lambertian  diffuse-reflection  equation  is  given  by: 

Id  =  IikdCOs{e)  (7) 
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i)  Diffuse 


b)  Specular 


Where  is  the  diffusion-reflection  coefficient  which  is  a  constant  between  0  and  1.  kj  depends  upon 
the  intrinsic  micro-structure  of  the  materied  surface. 

For  the  case  of  spectrcil  (light  frequency)  analysis,  I4  Ccin  be  decomposed  into  the  primary  color  com¬ 
ponents  as  follows: 


rred 

- 

I,k:t‘'cos(6), 

II 

lik^’''"'''cos(0), 

(8) 

II 

hk'i^'^cosie)- 

where  is  the  diffusion-reflection  coefficient  corresponding  to  each  color 

If  the  whole  visible  spectrum  is  to  be  considered,  equation  7  can  be  extrapolated  to  the  continuous 
wave  length  intervzd. 

h{\,e)  =  i,[\)k^{\)cos{d),  (9) 

where  kd{^)  is  the  diffusion-reflection  coefficient  as  a  function  of  the  wave  length  A  of  the  light  source. 

Specular  Reflection:  Any  shiny  smrface  (e.g  a  mirror)  exhibits  specular  reflections.  These  particular 
reflections  produce  a  highlight  spot  on  the  ob  k-ci  surface.  Such  a  reflection  is  a  function  of  the  view  point, 
the  light  somce  location  and  the  topology  '  *’  surface.  The  color  of  the  specular  reflection  does  not 
in  general  resemble  the  object’s  color,  the  light  somce  color  dominates  the  specular  reflection.  A  perfect 
mirror  will  only  reflect  light  on  the  direction  Lr  (See  Figure  28b).  A  viewer  will  be  able  to  observe  the 
specular  reflection  if  his/her  vantage  (view)  point  is  on  the  line  generated  by  Lr-  In  this  case  a  =  0,  and 
the  speculcir  reflection  is  just  the  image  of  the  light  source. 

A  model  for  specular  reflection  was  developed  by  Warnock  [71]  and  later  expanded  by  Phong  [8](who 
gave  his  name  to  the  model).  The  Phong  model  assumes  that  the  maximum  intensity  of  the  specular 
reflection  occxirs  when  a  =  0  and  falls  very  fast  as  a  increases.  The  intensity  decay  is  assumed  to  be 
proportional  to  co3"(a),  where  n  is  the  specular-reflection  exponent  of  the  material.  The  values  of  n  vary 
between  1  and  several  himdreds.  Large  n  provide  a  sharp  focused  highlight  and  small  values  correspond 
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to  a  broad  gentle  one.  A  perfect  mirror  cited  above  has  n  =  oo.  Figure  28c  shows  several  plots  of  cos"(a). 
The  intensity  of  the  specular  reflected  light  given  by  Phong’s  model  is: 

Is(X,e,a)  =  I,(X)W,(0)cos'^a  (10) 

where  is  the  object’s  specular-reflection  function  which  in  general  depends  on  the  incident  light  angle 

e. 

Ambient  Light:  There  is  another  component  of  the  reflected  intensity.  The  ambient-reflection  which 
is  due  to  the  diffuse  ambient  light.  This  reflection  is  easily  modeled  by: 

/  =  laK  (11) 

where  ka  is  the  object’s  intrinsic  ambient-reflection  coefficient  and  is  the  ambient  light  intensity  assumed 
to  be  consteint  for  zdl  objects  regeirdless  of  position  or  direction. 

Total  Reflection:  The  overall  reflection  intensity  is  obtained  from  the  superposition  of  the  diffuse- 
reflection,  spectdar-reflection  and  ambient  light  intensities  given  by  equation  9,  10  and  11  as  follows: 

Ir{Ke,a)  =  lak,  -i-  Ii{\)[kd{X)cos{e)  l^(0)cos”(a)]  (12) 

Ir  is  the  reflected  intensity  which  is  a  function  of  the  variables  A,  9  and  a 

The  environment  lighting  condition  given  by  the  ambient  light  term  laka  introduces  a  constant  term 
that  will  be  irrelevant  for  the  analysis  of  the  model.  Hence,  such  term  will  be  dropped  from  now  on. 

The  reflectance  function  is  defined  as: 

$(A,  0,  a)  =  [A:d(A)cos(0)  +  lF(^)co3"(a)j.  (13) 

so  that  the  reflected  intensity  is  given  by: 

/,(A,0,Q)  =  /,(A)$(A,0.a),  (14) 

The  reflectance  function  can  be  divided  into  two  main  components: 

§(A,0,a)  =  $,i(A,0)-f§,(A,0,a),  (15) 
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where  $(j(A,0,  a)  is  the  diffuse-reflection  function  given  by: 


^d(A,e)  =  MAWfl),  (16) 

and  #a(A,0,  a)  is  the  specular-reflection  function  given  by: 

$,(A,0.a)  =  W^(«)cos"(a).  (17) 

The  reflected  intensity  can  be  finally  written  as  the  superposition  of  the  two  types  of  reflection. 

Ir(XJ,a)  =  Li{X,S)  +  (18) 

where  the  diffuse-reflection  intensity  is 

I4X,0)  =  Ir{X)^4(X,0)  =  Ir{X)k4X)cos(e),  (19) 

and  the  specular-reflection  intensity  is  given  by 

I,{X,e,a)  =  IriX)is(XJ,a)  =  /,(A)ir(0)cos"(Q).  (20) 

5.3  The  Defect  Detection  Model 


The  defect  detection  model  discussed  here  integrates  two  techniques  to  isolate  defects  from  the  normal 
surface.  For  the  purposes  of  this  report,  these  techniques  are  given  the  names  color-subtractive  scheme  and 
multiple-illumination  scheme. 

Figure  29  shows  a  schematic  of  the  color-subtraction  scheme.  The  system  consists  of  one  single  light 
source  (white),  a  color  filter,  a  pouch  eind  a  camera.  The  image  registered  by  the  Ccimera  is  considered  to 
be  made  of  reflected  light  from  the  light-source  by  the  pouch  surface. 

Without  the  filter  (white  light  illumination),  the  defect-free  areas  present  diffuse-reflections  (mostly  of 
the  pouch  color:  green)  and  specular-reflections  (of  the  same  color  as  the  light  source:  white).  Speculcir- 
reflections  produce  shiny  spots  that  cire  function  of  the  surface  irregularities,  the  light  source  and  camera 
position. 
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Defective  regions  of  the  pouch  contribute  to  the  image  with  mostly  diffuse-reflections  and  occasionally 
(for  very  peirticular  conditions)  specular-reflections.  In  any  case,  defective  regions  produce  reflections  of 
the  same  color  as  the  light  source.  That  is  because  the  defective  areas  are  assumed  to  be  regions  where 
the  metal  coating  is  exposed. 

The  color-subtractive  scheme  consists  of  illuminating  the  pouch  with  the  colored  light  so  that  the 
diffuse-reflected  green  light  of  the  defect-free  region  is  almost  completely  suppressed  from  the  image.  If 
the  light  is  filtered  so  that  the  incident  colored  light  lacks  the  green  component,  the  defect-free  areas  will 
absorb  most  of  the  incident  light.  This  is  made  possible  by  using  a  magenta  colored  filter. 

Prom  the  mathematical  point  of  view  the  color-subtractive  scheme  is  easily  explained.  In  equation  19 
the  color  distribution  of  the  incident  light  Ii(X)  is  minimized  on  the  values  of  A  where  the  diffuse-reflection 
function  $d(A,0)  of  the  material  is  maximum.  That  is  the  region  of  the  spectrum  corresponding  to  the 
green  color. 

Since  the  filter  is  not  perfect  (a  perfect  magenta  filter  has  no  green  component  at  aU)  some  residual 
diffuse-reflection  is  present  in  the  image.  However,  the  residual  intensity  is  considerably  lower  than  the 
intensity  of  defective  regions. 

Colored-subtractive  schemes  isolate  and  reduce  the  contribution  of  diffused-reflection  from  defect-free 
regions.  However,  the  specular-reflection  from  these  regions  and  the  reflection  from  defects  are  of  the  seune 
color  as  that  of  the  incident  light.  Therefore,  they  caimot  be  isolated  by  a  color-subtractive  scheme.  The 
goal  of  the  multiple-iUumination  scheme  is  to  separate  these  two  components. 

As  described  above,  the  specular-reflections  given  by  equation  20  are  functions  of  the  surface  shape 
and  the  relative  position  of  the  light  source  and  camera.  Assuming  that  the  surface  shape  and  the  ceimera 
position  remain  const  ant,  then,  as  the  light  source  moves  from  one  place  to  another,  the  relative  position 
of  the  bright  spot  corresponding  to  specular-reflection  moves  as  well. 

If  several  images  eire  registered,  corresponding  to  different  iUiunination  configurations,  it  is  possible  by 
processing  the  complete  set  of  images  to  separate  the  contribution  of  the  specular-reflection.  The  main 
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issue  in  this  technique  is  to  use  the  fact  that  the  pattern  of  specular-reflection  is  a  function  of  the  hght 
source  position.  Since  the  placement  of  the  hght  source  does  not  change  the  object-image  mapping  (which 
is  a  function  of  the  camera-object  position),  it  is  possible  to  easily  correlate  seve^^ll  images  with  different 
hght  conditions. 

The  bright  spot  in  the  image  corresponding  to  defective  regions  appears  in  all  the  images.  This  is 
because  the  contribution  of  defective  regions  is  mostly  due  to  diffuse-reflections  whose  reflection  functions 
chzinge  slowly  for  smadl  values  of  5  as  given  by  equation  19.  On  the  other  heind,  the  specular-reflection 
given  by  equation  20  is  represented  by  a  sharp  function  of  the  hght  source-camera  phase  angle  a.  Prom 
Figure  28  it  is  clear  that  A6  =  Aa  so  that  small  changes  in  the  hght  source  position  will  produce  Icirge 
changes  in  the  specular-reflection  pattern  while  the  diffuse-reflection  pattern  remains  constant. 

Figure  30  shows  several  images  that  demonstrate  the  principles  discussed  in  this  section.  Figure  30. A  is 
a  gray  level  image  of  a  pouch  area.  Figures  30.B  and  30. C  are  two  images  of  the  same  pouch  with  different 
illumination  conditions.  It  is  cleeir  that  the  pattern  of  specular  reflection  is  very  different.  Figure  30. D  if 
the  final  result  (after  the  “AND”  process)  for  a  set  of  seven  images.  In  this  case  one  defect  was  detected. 
It  is  clear  that  the  defective  area  is  present  in  ah  of  the  images. 

5.4  Alternative  System 

Without  going  into  mzmy  details  it  is  possible  to  use  the  same  reasoning  presented  in  5.2  to  obteiin  an  eil- 
temative  system  configuration.  This  alternative  configuration  was  motivated  by  implementation  problems, 
since  it  was  fotmd  difficult  to  filter  each  single  hght  source. 

The  principal  implementation  problem  was  that  the  hght  soxirce  (halogen  hght  bulb)  released  iMge 
amoimts  of  heat  that  no  filter  could  withstand.  This  could  be  compensated  by  adding  coohng  systems, 
but  implementation  then  would  be  more  expensive  emd  complex. 

The  same  reflection  model  used  earher  aUows  the  derivation  of  an  improved  model  which  performs 
better  than  the  initial  one;  it  is  also  easier  to  implement  and  more  immime  to  ambient  hght  perturbation. 
Figure  31  shows  a  detection  system  based  on  a  filtered  camera  with  a  white  incident  hght  source. 
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Figure  31;  Defect  detection  system:  Filtered  camera 


Pouch 


When  using  a  white  light  source  the  light  reflected  by  the  pouch  is  heavily  loaded  with  green  components 
especizdly  from  those  areas  that  eire  neither  defects  nor  speculzu-reflections.  Defective  areas  and  specular- 
reflections  reflect  mostly  light  of  the  S2ime  color  them  the  light  source  (white). 

If  a  magenta  filter  is  placed  in  front  of  the  camera,  after  the  reflected  light  has  passed  the  filter,  the 
green  component  of  the  pouch’s  normal  surface  is  removed  by  the  same  color-subtractive  scheme  discussed 
eeirlier.  The  white  light  coming  out  of  the  defective  reg*  and  the  specular-reflection  turn  magenta  because 
of  the  filter. 

As  shown  in  Figure  31  the  two  cases  of  defect-free  and  defective  regions  have  the  satme  color  spectrum 
at  the  C2imera  position  as  the  one  presented  in  Figure  29.  So,  this  detection  system  (called  filter-Ccimera) 
is  equivalent  to  the  initial  one,  however,  it  is  easier  to  implement  since  there  is  only  one  filter. 

5.5  System  Prototyping 

5.5.1  The  Lighting  Structure 

Given  the  analytical  background,  prototyping  of  the  system  resulted  in  a  structure  called  “The  Dome”. 
Figure  33  shows  a  pictiue  of  the  actual  implementation.  ‘The  Dome”  is  a  hemisphere-like  structure 
designed  so  that  the  camera  is  located  at  its  zenith.  Figure  32  shows  a  schematic  representation  of  a  seven 
light-source  configuration.  They  axe  located  along  two  meridian  semicircles  symmetrically  placed  at  15°  of 
each  side  of  the  camera  meridian.  Along  each  of  these  two  meridians  4  and  3  light  sources,  respectively, 
are  uniformly  distributed  at  20°  intervals. 

The  camera  used  in  this  implementation  had  a  512x512  CCD  array.  It  was  configured  with  a  16mm 
1:1.4  lens,  10mm  spacer  and  x2  extender.  The  C2unera  was  placed  30cm  from  the  pouch  wuh  a  diaphragm 
aperture  of  1.4.  For  this  set-up  a  image  is  a  projection  of  a  3. 2x3. 2  cm  region  on  the  pouch,  hence,  the 
resolution  of  the  system  is  l/16mm  x  1/16  mm  per  pixel.  The  light  somces  used  were  halogen  12V  SOW. 
This  type  of  light  is  a  good  source  of  white  light. 

The  overall  system  configmation  is  shown  in  Figure  34.  A  PC  computer  drives  the  light  array  and 
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Figure  33;  Prototype  of  the  vision  system 
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Figure  34:  Machine  vision  system  configuration 


interfaces  with  the  CCD  camera  controller.  After  the  seven  images  are  digitized  by  the  PC  the  information 
is  then  transferred  (via  ethemet  network)  to  a  SUN/3  workstation  which  nms  the  image  processing  code. 

A  program  running  in  the  PC  interfaces  with  the  ceunera  card  controller  to  digitize  images.  Each  image 
is  initially  stored  in  a  256KB  BAM  memory  buffer  and  later  transferred  to  a  file  in  the  HD.  Before  an  image 
is  taken,  the  program  interfaces  through  the  parallel  port  (Centronix  protocol)  with  the  specially  designed 
light-array  controller  to  switch  each  individual  light  on/off.  In  this  way,  seven  images  are  registered  each 
one  with  a  different  lighting  direction. 

A  block  diagram  of  the  light-array  interface  controller  is  shown  in  Figure  35.  It  uses  a  8-bit  parallel- 
latch  which  uses  the  Strobe  signal  of  the  parallel  port  to  latch  a  data  byte.  Each  one  of  these  signals  drives 


86 


PC  COMPUTER 


Parallel  Port  Data  Bus  D0-D7 
The  STROBE  signal 


D7 


1-Bit 

LATCH 


U1:74LS374 
T:  PNP2907 
RL:  SPDT  12V  4A 
R»:  IK 

L:  Halogen  12V  SOW 


+12V 


RL7 


Figure  35:  Light  array  interface 


a  PNP  2N2907  transistor  which  in  turn  is  used  to  drive  a  power  relay.  The  set  of  7  relays  drives  the  light 
bulbs  directly.  The  function  of  this  power  amplifier  chain  is  to  convert  the  5  volt  10  mA  (50  mW)  signal 
of  the  par2dlel  port  into  a  12  volt  4A  (50  W)  required  for  the  halogen  light  bulbs. 

5.5.2  The  Image  Processing 

Image  Segmentation:  Once  the  N  gray  level  images  of  the  same  pouch,  under  different  light  directions, 
have  been  obtained,  they  are  thresholded  (binarized).  A  local  thresholding  technique  must  be  used  since 
specular-reflection  can,  in  many  cases,  be  brighter  than  reflection  from  small  defective  regions.  In  these 
cases,  defective  regions  can  be  driven  to  background  levels  n  a  global  thresholding  technique  is  used. 

The  segmentation  software  divides  the  image  into  a  block  of  m  x  m  pixels.  Each  block  is  binzurized 
with  its  own  local  threshold  value.  To  do  this,  the  histogram  for  each  block  is  constructed  and  smothered. 

Image  Integration:  The  set  of  thresholded  images  undertakes  an  AND  process  to  determine  whether 
a  region  is  present  in  aU  the  images.  As  it  was  discussed  before,  the  source  of  such  a  region  belongs  to 
cireas  on  the  pouch  that  diffusely  reflect  light  from  every  light  source;  and  is  most  likely  a  defective  area. 
An  alternative  result  “iV-f  out  of  N'  is  edso  computed  so  that  the  consistency  of  the  AND  resxilt  can  be 
evaluated.  The  “A-J  out  of  N"  is  a  process  that  determines  the  pixels  that  appear  on  N-1  images  out  of 
the  N  images. 

5.6  Conclusions 

The  vision  system  is  the  most  critical  component  of  the  MRE  robot  based  inspection.  It  is  important 
to  design  a  system  which  is  reliable  and  efficient.  The  prototype  system  was  trained  on  one  set  of  MRE 
pouches  and  tested  on  a  second  set  of  pouches.  Both  the  training  and  test  sets  were  10  MRE  pouches  of 
Ham  slices. 

For  the  prototype  developed,  a  5%  error  rate  was  obtained.  That  is,  5%  of  the  true  defects  were  missed; 
however,  no  defective  pouches  were  missed.  We  found  that  most,  if  not  edl  of  the  test  defective  pouches 
had  more  than  one  defect  and  in  all  cases  at  least  one  of  the  defects  was  detected.  The  technique  proved 


88 


to  work  in  general  and  the  result  is  good  for  a  first  prototype  system.  Furthermore,  none  of  the  normal 
pouches  were  incorrectly  classified  as  defective  by  the  system. 

Although  this  is  promising,  caution  should  be  exercised  and  more  extensive  testing  needs  to  be  done 
on  MRE  pouches  filled  with  other  material  that  causes  the  surface  not  to  be  flat.  As  it  was  discussed 
before,  the  specular-reflection  produced  highlighted  areas.  For  ideal  mirror-like  surfaces,  these  regions  are 
concentrated  in  small  areas.  There  positions  change  drastically  as  the  light  source  moves.  For  regrilar 
surfaces,  these  reflections  form  large  highlighted  spots.  If  the  light  source  position  changes  by  a  small 
amount  (bright  field  of  illumination)  it  is  possible  that  there  will  exist  overlapping  regions  that  appeeir 
on  all  the  images.  This  is  because  of  the  size  of  the  specular-reflection;  some  regions  present  considerable 
specular  reflection  under  all  the  set  of  light  sources  used  by  the  multiple-illumination  scheme. 

A  schematic  representation  of  this  phenomenon  is  presented  in  Figure  36a.  When  a  bright  illumination 
field  is  used,  it  is  possible  that  the  two  (or  more)  specular-reflection  highlights  have  a  region  were  they 
overlap.  If  the  intensity  of  the  region  is  such  that  several  of  the  image’s  pixels  are  digitized  to  a  foreground 
level,  it  is  possible  that  a  small  set  of  pixels  survive  the  AND  process.  In  this  case  a  defect-free  area  will 
be  classified  as  a  defective  one. 

The  system  cm  be  improved  if  a  dark  field  of  illumination  is  used.  Figure  36b  shows  such  a  situation 
for  the  case  of  two  light  sources.  The  two  highlighted  spots  (corresponding  to  the  two  light  sources)  are 
separated  enough  so  that  the  intensity  of  my  overlapoing  region  is  small.  In  that  case  the  probability  of 
my  overlapped  region  surviving  the  AND  process  is  very  small.  Several  experiments  were  performed  where 
only  four  light  sources  distributed  in  a  large  iUiuninetion  field  were  used  (  more  thm  50°  with  respect  to 
the  pouch  normal).  Even  though  the  false-positive  detection  weis  decreased,  two  new  problems  appeared 
for  this  system. 

If  the  incident  light  is  placed  at  a  large  mgle  with  respect  to  the  sTirface  normal,  the  diffuse-reflection 
component  decreases  as  given  by  equation  5.3.  Most  of  the  light  reflected  out  of  a  defective  area  is  diffused, 
which  mems  that  for  these  illumination  conditions  the  image  corresponding  to  the  defective  aireas  will  be 
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Figure  36:  Dairk  vs  Bright  illumination  fields 
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Figure  37;  Hidden  defective  area 

very  dark  (close  to  background  level).  In  this  case  they  are  digitized  to  background  and  will  not  survive 
the  AND  process.  In  other  words,  defects  will  not  be  detected.  The  problem  of  low  intensity  reflections 
at  large  incident  illumination  amgles  can  be  easily  solved  if  the  intensity  of  the  incident  light  source  is 
increased  for  those  large  angles.  The  intensity  can  be  increased  by  using  more  powerful  light  sources  or 
just  by  placing  the  light  source  closer  to  the  pouch. 

Another  problem  that  could  appear  is  shown  in  Figure  5.6  where  a  defect  region  is  hidden  from  the 
incident  light  due  to  a  large  surface  irregularity.  The  problem  becomes  more  notable  when  a  leurger  field 
of  illumination  is  used.  A  solution  to  this  problem  is  to  stretch  out  the  pouch  so  that  large  wrinkles 
are  flattened.  The  previous  section  described  an  automatic  device  that  performs  such  an  operation  (the 
stretching  table). 

The  image  acquisition  time  is  currently  7  seconds  and  the  image  processing,  which  is  pipelined  and 
parallel  with  pouch  transport  is  7  seconds.  The  processing  time  cein  be  improved  if  pEirzillel  computer 
hcirdwzire  is  used.  The  segmentation  of  each  of  the  image’s  freimes  cem  be  carried  in  paxedlel.  A  dedicated 
set  of  image  processors  C2in  be  designated  to  implement  such  an  operation.  The  images  cam  be  held  in 
256K  RAM  memory  buffers  attached  to  each  processor  to  decrease  the  access  time. 

The  thresholding  of  the  first  image  can  begin  as  soon  as  that  image  has  been  captured.  While  that  first 
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processor  is  segmenting  the  first  image,  the  camera  can  be  loading  image  heunes  into  the  other  processor 
buffers.  It  has  been  estimated  that  the  total  processing  time  will  not  exceed  7  seconds.  Currently  the 
protot}rpe  process  takes  seversd  minutes,  this  is  due  to  the  data  transfer  eunong  machines  and  the  seried 
processing  of  the  seven  frames  by  the  SUN3  workstation. 
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6  Economic  Analysis 

6.1  Economic  Analysis  of  Robotic  Workcells 

A  computer  program  for  economic  analysis  of  automated  workcells  (EARW)  was  developed  by  modifying 
an  existing  program  wrritten  by  Feing  [25].  The  description  of  this  program  is  given  in  the  following; 

The  procedures  used  in  this  analysis  allow  the  summing  of  funds  that  occur  at  different  times.  Six 
indicators  axe  produced  for  evaluating  the  economic  performance  of  any  user  defined  cases.  They  are: 

1.  Project  Life  (L), 

2.  Years  of  Operation  (N), 

3.  Expected  Return  on  Investment  (EROI), 

4.  Return  on  Investment  (ROI), 

5.  Cumulative  Present  Worth  (CPW),  and 

6.  Payback  Period  (YE). 

They  can  also  be  used  for  comparison  between  different  cases. 

The  amounts  of  funds  for  operation  to  be  recorded  are  as  follows; 

1.  Expense:  fixed  cost  +  operating  cost 

2.  Revenue:  the  saving  of  reduced  labor  eind  Vcilue  of  eliminated  manual  system  devices 

3.  Return:  revenue  -  expense 

Every  record  represents  the  situation  of  the  flow  of  fimds  at  the  end  of  each  year,  starting  from  year  0 
(which  is  the  beginning  of  the  project).  The  record  of  the  zeroth  year  is: 

1.  Expense:  the  initial  cost  of  robotic  workcell  (I) 
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2.  Revenue:  0 


3.  return:  -I 

Tetking  the  starting  time  of  the  project  as  “present  time”,  the  worth  of  the  end  of  yezu:  i  will  be 
reflected  to  its  “present  worth”  by  dividing  it  by  (1  +  EROI)*.  The  sum  of  aU  the  “present  worth”  from 
year  0  through  N  becomes  the  CPW. 

EROI  is  an  estimated  value  provided  by  the  investor  and  ROI  is  the  return  rate  on  an  investment  over 
the  lifetime  of  the  project  (i.e.,  when  N=L).  Payback  period  (YE)  is  calculated  based  on  EROI,  that  is, 
the  time  required  for  achieving  the  expected  return  on  investment.  When  EROI  is  zero,  YE  becomes  the 
simple  payback  period  (YS). 

The  following  rules  are  used  to  interpret  the  result  of  this  economic  analysis: 

1.  If  “N  =  L  and  CPW  >  0”  then  “ROI  >  EROI” 

2.  If  “N  =  L  and  CPW  <  0”  then  “ROI  <  EROI” 

3.  If  “N  =  L  and  CPW  =  0”  then  “ROI  =  EROI” 

4.  If  “N  <=  L  and  CPW  =  0”  then  “YE  =  N  with  a  return  equal  to  EROI” 

5.  If  “N  <=  L  and  CPW  =  0  and  EROI  =  0”  then  “YS  =  N  with  0  return” 

The  initial  cost  in  this  project  is  the  cost  of  automated  workcells.  And,  the  revenue  is  labor  savings, 
that  is,  savings  of  salzuy  plus  associated  expenses.  The  economic  Einalysis  presented  in  this  report  is  based 
on  the  above  concepts. 

The  computer  program  (EARW)  calculates  a  set  of  dependent  variable  vedues  based  on  the  user-defined 
independent  variable  values.  EARW  is  a  QuickBasic  based  program  and  its  compiled  version  will  run  on 
most  of  DOS  based  microcomputers.  The  source  code  of  EARW  is  listed  in  the  Appendix.  There  2ire  three 
categories  of  independent  variables.  The  independent  variables  and  their  base-line  values  will  be  discussed 
later. 
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The  cycle  time  is  the  average  time  required  to  finish  one  product.  The  task  inefficiency  indicates  the 
percentage  of  unsuccessful  cycles.  The  busy  ratio  represents  the  percentage  of  real  working  hours  spent  on 
production  lines. 

The  dependent  variables  and  their  units  are: 

1 .  first  yecir  return  in  dollars 

2.  cumulative  present  worth  in  dollars 

3.  ciunulative  present  worth  in  d'-’lars  per  1000  units  of  product 

4.  simple  payback  years,  in  years 

5.  expected  payback  ycMs,  in  years 

6.  return  on  investment,  in  % 

After  entering  the  independent  variable  values,  the  program  has  three  functions  for  economic  analysis: 

1.  feasibility  einalysis, 

2.  parametric  einalysis,  and 

3.  comparison  of  alternatives. 

6.1.1  Feasibility  Analysis 

For  a  particidar  design,  its  economic  feasibility  will  be  evaluated  usmg  the  manual  operation  as  a  refer¬ 
ence.  Therefore,  the  result  eilways  shows  the  comparison  between  the  automated  workceU  and  the  m^lnual 
operation  based  on  the  independent  variables  specified  by  the  usei.  The  result  shows  the  interrelations 
among  annual  work  volume,  required  daily  productivity,  number  of  working  days,  corresponding  requir  ed 
number  of  workers  and  workcells,  YE  and  ROI.  The  required  daily  productivity  is  the  dominating  factor 
which  decides  the  number  of  hurnam  workers  or  automated  workcells  needed  to  perform  the  teisk. 
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6.1.2  Parametric  Analysis 


This  part  of  program  reveals  the  effects  of  variations  of  dependent  variables  on  independent  variables. 
During  each  program  execution,  one  dependent  variable  and  two  independent  variables  cein  be  displayed 
on  a  graph.  The  dependent  Vciriable  is  shown  as  the  vertical  eixis,  one  of  the  independent  variables  is  shown 
as  the  horizontal  auds  and  the  other  one  as  the  control  variable.  The  user  can  select  the  variables  and  define 
their  ranges.  As  a  result,  a  family  of  curves  are  drawn  showing  the  interdependence  of  the  three  variables 
involved. 

B^'  this  analysis,  the  following  are  example  questions  which  cam  be  answered: 

1.  How  low  should  the  price  of  workcell  be  in  order  to  obtjun  at  least  a  given  level  of  KOI? 

2.  How  much  more  may  an  automated  workcell  cost  for  a  desired  increase  in  productivity? 

3.  How  many  hours  should  the  automated  system  work  so  the  payback  period  is  within  a  certain  number 
of  years? 

6.1.3  Comparison  of  Alternatives 

EARW  can  compare  up  to  10  different  workcells  at  a  time.  Therefore,  up  to  10  sets  of  vJues  related 
to  automated  tzuik  can  be  entered  by  the  user.  The  result  of  comparison  will  be  shown  for  a  chosen 
independent  variable  (as  the  horizontal  2ixis)  and  a  chosen  dependent  veiriable  (as  the  verticcd  2ixis).  This 
function  of  EARW  was  not  used  in  this  study. 

6.2  Results  of  Economic  Analysis 

6.2.1  Workcell  costs  versus  human  inspector  labor  cost 

The  cost  of  robotic  workcell  was  estimated  by  the  retail  prices  listed  in  industrial  catalogs  and  through 
telephone  survey.  The  items  which  were  not  standard  off-  the-shelf  products,  such  as  the  end-effector, 
the  estimation  was  based  on  the  components  to  be  used  and  the  cost  of  development.  For  the  purpose 
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of  comparison,  two  types  of  automated  inspection  workcell  were  studied:  flexible  automation  and  hsud 
automation. 

The  estimated  costs  for  the  components  of  the  inspection  workcells  are  given  in  Section  4.  The  total 
costs  of  flexible  and  hard  automation  inspection  workcells  were  estimated  to  be  $56,500  2ind  $40,000, 
respectively.  These  estimated  costs  were  based  on  the  situation  when  a  small  number  of  workcells  were 
purchased.  When  a  leirge  number  of  workcells  are  needed,  it  is  conceivable  that  the  real  costs  could  be 
substeintially  lower  thcin  these  estimated  costs.  During  economic  analysis,  a  price  range  of  25  to  60  thousand 
doUeurs  was  used. 

The  labor  cost  for  human  inspectors  was  estimated  to  include  an  hourly  pay  of  $7.50  per  hour  Eind  an 
indirect  cost  of  $5.00  per  hour;  therefore,  a  total  cost  of  $12.50  per  hour  was  used  as  the  baseline  vedue. 

6.2.2  Equivalent  capacity  study  (automated  workcell  versus  human  inspector) 

One  purpose  of  the  economic  analysis  is  to  determine  the  number  of  automated  workcells  required  to  keep 
the  s^une  pleint  throughput  as  the  plant  with  the  hiunan  inspectors.  That  is,  if  the  production  rate  is  held 
consteint  then  the  eqtiivalent  number  of  automated  workcells  depends  on  the  lumped  characteristics  of  the 
total  number  of  humein  inspectors.  The  economic  analysis  model  EARW  calculates  the  number  of  workcells 
on  the  basis  of  daily  productivity,  work  horns,  task  inefficiency,  and  busy  ratio.  This  calculation  gives  a 
conservative  estimate.  If  a  more  realistic  estimate  is  needed,  the  numerical  simulation  model  should  be 
used. 

One  cautionary  note  is  that  this  result,  while  important  to  consider,  is  not  complete  since  defect 
detection  rates  cire  not  included  as  part  of  this  model.  On  the  basis  of  total  throughput,  many  more 
workcells  zire  needed,  however,  when  one  considers  defect  detection  (as  part  of  complete  feasibility)  there 
is  not  such  as  dr2miatic  difference. 

The  number  of  units  produced  per  hoiu  per  mm  is  394  and  the  munber  of  units  produced  per  hour 
per  workcell  is  234  to  yield  the  daily  production  of  100,000  units.  These  statistics  are  computed  from  the 
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following  model,  derived  from  our  survey  of  existing  plants: 


1)  Conditions  - 

daily  production  rate  =  100,000  pouches 
daily  work  period  =  10  hours 
manual  task  inefficiency  —  1% 
manual  busy  ratio  =  87.5% 
automated  task  inefficiency  =  4% 

2)  Equivalent  numbers  - 
human  workers  =  22 
automated  workceUs  =  41 


6.2.3  Baseline  values  for  economic  analyses 

The  economic  analysis  was  conducted  in  three  parts.  The  first  two  were  wide-range  feasibility  and  para¬ 
metric  anedyses,  one  for  flexible  automation  and  the  other  one  for  hard  automation.  The  third  part  weis 
conducted  to  demonstrate,  for  either  type  of  automated  workcell  replacing  both  pre-retort  and  post-retort 
inspection  workers,  the  influence  of  workcell  cost,  work  hours  per  day,  and  zinnual  work  volume  on  the 
simple  payback  years. 

Figures  38  amd  39  show  the  baseline  values  for  engineering  economic  aneilysis  for  flexible  automa¬ 
tion  emd  hard  automation,  respectively.  These  baseline  values  were  derived  from  our  survey  of  current 
production  facilities.  The  assumptions  for  these  baseline  values  are  as  follows: 

1.  The  inspection  time  for  a  robotic  workcell  is  the  same  for  both  pre-retort  sind  post-retort  inspection 
lines,  that  is,  we  assume  that  the  workcell  c^ul  do  either  job  in  the  same  time  period. 
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N6na  of  dunging  Par«feten 


Irhn  fnllnuinn  cidliies  are 

1.  Labor  Costs. 

$/hr 

12.5 

Hanual 

2.  Task  Cycle  Tim. 

sec/tinit 

7 

Task  - 

3.  Hbrking  hours  per  day. 

hr 

IB 

4.  Task  inefficiency. 

declml 

.81 

5.  Busy  ratio. 

decimi 

.075 

6.  Ubrkcell  Costs. 

1EGB$ 

56.5 

Auto 

7.  X/HCosts 

X  of  IteM  6 

12 

Task  - 

a.  Task  Cycle  Tim. 

sec/Unit 

14 

9.  Uorking  hours  per  day. 

hr 

IB 

fl.  Task  inefficiency. 

decimi 

.Bt 

1 

B.  Annual  Ubrk  Lblum. 

IBGB  Units 

288GB 

General 

C.  Required  Unrk  rate. 

lEOB  Unlts/iday 

1GB 

info. 

D.  Exited  Return  On  Inuestmnt 

decimi 

B 

1 

E.  Exited  Unrkceli  LifeTim. 

year 

IB 

Hit  1  No.  to  change  or  (REIURl)  to  leave. 


Figtire  38;  EARW  table  showing  parameters  of  flexible  automation. 


^  of  Changing  Parafetera 


'[Tie  foilouiiiq  wines  are  set. . . 
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7 
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■ 
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hr 

IB 
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r6. 
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48 
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7. 
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12 

Task 

— 

8. 

Task  Cycie  Tim. 
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14 

9. 

Harking  hours  per  day, 

hr 

IB 

^  A. 

Task  Inefficiency, 

decimi 

.84 

1 

B. 

Annual  Hark  U)lum. 

IGGBUniU 

28GGB 

General 

H 

C. 

Required  Hark  rate. 

Existed  Return  On  Investment 

IGGB  Unlts/day 

1GB 

info. 

B. 

decimi 

8 

\ 

E. 

Expected  Uarkcell  llfeTim. 

year 

IB 

Hit  a  Ho. 

to  change  or  <REIliRN>  to  leave. 

Figure  39:  EARW  table  showing  parameters  of  hard  automation. 
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2.  The  task  inefficiency  of  a  robotic  workcell  represents  the  combination  of  defect  detection  errors  in 
the  image  walysis  and  failures  due  to  pouch  mishandling.  Although  they  aire  lumped  together,  the 
majority  of  the  inefficiency  is  assumed  to  be  due  to  mishandling,  based  on  our  image  analysis  results 
in  Section  5. 

3.  The  task  inefficiency  of  human  inspectors  is  the  percentage  of  false  decision  meiking. 

4.  The  number  of  working  days  in  one  year  is  280  days. 

5.  The  lifetime  of  a  workceU  is  10  years. 

Six  cases  of  economic  analysis  were  conducted.  They  were; 

1.  flexible  automation  workcell  (FAW)  versus  human  inspectors  for  pre-retort  lines, 

2.  hard  automation  workcell  (HAW)  versus  human  inspectors  for  pre-retort  lines, 

3.  FAW  versus  human  inspectors  for  post-retort  lines, 

4.  HAW  versus  human  inspectors  for  post-retort  lines, 

5.  overall  analysis  of  FAW,  eind 

6.  overall  emalysis  of  HAW. 

For  each  czise,  we  performed  both  a  pessimistic  and  an  optimistic  analysis  to  determine  the  range  of 
results.  For  cases  5  cind  6,  we  adso  performed  the  baseline  case  analysis.  The  pessimistic  cases  are  those  with 
the  best  scenarios  of  human  inspectors  versus  the  worst  scenarios  of  automated  workcell,  and  the  optimistic 
case  were  the  best  scenarios  of  automated  workcell  versus  the  worst  scenarios  of  human  inspectors. 

6.2.4  Feasibility  analysis 

It  was  found  that  in  order  to  inspect  28  million  pouches  per  year  on  the  pre-retort  Unes,  13  workceUs  were 
required  to  replace  8  human  inspectors  if  the  workcells  were  operating  16  hours  per  day.  In  this  pessimistic 
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case  for  FAW,  the  investment  would  be  paid  back  in  12.104  years.  The  ROI  was  less  than  zero  because  the 
payback  period  is  longer  than  the  10  years  project  lifetime. 

In  the  optimistic  C2«e,  for  FAW  to  replace  pre-retort  inspectors,  eight  workcells  were  required  to  replace 
8  inspectors  based  on  a  24  daily  working  hour  schedule  for  the  workcells.  The  simple  payback  period  was 
1.276  years  eind  the  ROI  for  the  project  (over  system  life  time  of  10  years)  was  78.127%. 

In  the  pessimistic  case  for  FAW  to  replace  post-retort  inspectors,  thirteen  workcells  were  required.  The 
simple  payback  period  was  3.546  years  and  the  ROI  was  25.251%. 

In  the  optimistic  case  for  FAW  versus  post-retort  inspectors,  nine  workcells  were  necessary  to  replace 
15  inspectors.  The  simple  payback  period  was  0.744  years  and  the  ROI  was  134.378%. 

For  the  analagous  situations  employing  HAW  for  the  pre-retort  lines,  ROI  was  less  than  0  in  the 
pessimistic  case.  The  result  of  the  optimistic  case  indicated  that  the  simple  payback  period  was  3.156 
years  and  the  ROI  was  29.251%.  For  the  post-retort  inspection  lines,  the  simple  payback  period  was  8.975 
years  for  the  pessimistic  case,  and  the  ROI  was  2.126%.  For  the  optimistic  case,  the  simple  payback  period 
was  1.56  years  and  the  ROI  was  63.626%. 

There  zire  2  ways  to  perform  economic  analysis  for  the  overall  inspection  lines  (considering  both  pre¬ 
retort  and  post-retort  lines  together): 

1.  Use  the  formula  which  gives  the  weighted  average  of  ROIs  for  pre-retort  and  post-retort  lines  to 
calculate  the  overall  ROI  values  from  the  results  of  sepzirate  pre-retort  lines  analysis  and  post-retort 
lines  analysis. 

2.  Directly  input  the  overall  task  cycle  time  as  7  seconds  (2.5  -|-  4.5)  for  human  inspectors  and  14 
seconds  (7-1-7)  for  automated  workcell  into  the  program  EARW  and  keep  all  other  values  the  same. 

The  result  of  overall  emalysis  showed  that,  in  the  pessimistic  case,  a  workcell  using  flexible  automation 
will  be  paid  back  in  5.485  ye£irs  zind  the  ROI  will  be  12.751%.  For  HAW,  the  payback  period  wiU  be  4.042 
years  and  the  ROI  will  be  21.126%  in  the  pessimistic  case.  It  also  showed  that,  in  the  optimistic  case,  both 
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FAW  and  HAW  were  going  to  give  us  108.002%  of  ROI.  For  the  baseline  (i.e.,  average)  case,  it  was  found 
that  ROI  would  be  52.126%  using  FAW  and  60.501%  by  using  HAW. 

6.2.5  Parametric  analysis 

This  pjirt  of  study  investigated  the  dependence  of  ROI  on  some  important  factors,  such  as  labor  costs, 
workcell  costs,  operation  and  maintenance  costs  (R/M  costs),  workcell  task  cycle  time,  and  work  horns  per 
day. 

In  one  example,  it  was  found  that  1)  if  the  workceU  cost  $50,000,  the  ROI  was  greater  them  45%  if 
labor  costs  were  higher  than  $ll/hr,  and  2)  if  the  labor  costs  increased  by  $l/hr,  an  investment  of  $5000 
more  coidd  be  justified,  maintaining  the  same  ROI.  Prom  all  the  parametric  studies  performed  using  the 
baseline  values  listed  in  Figures  38  and  39,  the  following  observations  were  made: 

1)  ROI  decreased  by  2%  as  R/M  costs  increased  by  1.5%  of  workceU  costs  when  labor  costs  were  $ll/hr. 

2)  The  effect  of  R/M  costs  decreased  when  labor  costs  increased. 

3)  ROI  and  workceU  cycle  time  (WCT)  are  inversely  related  and  that  relationship  is  nonlinear.  There¬ 
fore,  a  smaU  reduction  in  workceU  cycle  time  could  cause  a  rapid  increase  in  ROI. 

4)  ROI  would  increase  about  1%  if  workceUs  worked  one  more  hour  per  day. 

5)  If  a  workceU  cost  more  than  $60,000,  workceU  cycle  time  needed  to  be  less  thain  15  seconds  in  order 
to  have  higher  than  15%  ROI. 

The  result  of  this  parametric  analysis  not  only  provides  us  more  insights  about  the  feasibUity  of  the 
automated  inspection  workceUs  but  zdso  shows  which  part  of  the  workceU  is  the  most  influentied  on  ROI. 

Specific  cases: 

Figure  40  is  the  result  of  economic  analysis  using  the  baseUne  veilues  shown  in  Figure  38.  It  was 
estimated  that  41  automated  workceUs  were  needed  to  replace  22  human  inspectors  with  compeirable 
productivity  (100,000  pouches  inspected  per  day).  The  simple  payback  period  for  this  pzu'ticular  case  was 
3.459  years  and  the  overaU  return  on  investment  of  41  automated  workceUs  was  26.126%. 
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Based  on  the  glvee  walues  : 

Annual  ybrk  Uslum  2BGGB.8B  x  1EB9  units/\jear 

Nuidwr  of  VORKEB  auai  table  :  22.008 

Nufdier  of  IKOL  aueilable  :  dl.GGB 


Manual 

AutOMtlC 

Task  ve.  Task 

Given  lurking  hours/day 

18.88 

18.68  hrs/day 

Hriy  Unit  U^k  Rate 

445.588 

246.857  nnitsdir 

Dally  Unit  Uork  Rate 

4455.888 

2468.572  units/day 

Daily  Ubrk  Rate 

181.211  X  1888  units/day 

mifliier  of  DAVs  reqiuired 

285.685 

276.649  days 

Total  Labor  Cost 

X  1688$ 

Total  Fixed  Cost 

2316.580  xl888$ 

Total  Operating  Cost 

115.825  xie88$ 

Expected  Payba^  Vaar  *1 

3.459  years 

Return  On  Investient  *2 

26.126  X 

•1:  EROI  is  0.68  X  *2:  Systen  LifeTIxe  is  10  years 

Press  (Enter)  to  continee.. 


Figure  40:  EARW  output  for  flexible  automation  system. 

It  was  believed  that  the  $56,500  was  a  good  estimate  for  the  automated  workcell  designs  developed  in 
this  research.  However,  this  estimated  cost  does  not  consider  the  possibility  of  discoimted  price  normally 
anticipated  when  ordering  a  large  ntunber  of  workcells  at  one  time.  The  effect  of  workcell  cost  is  shown  in 
combination  with  annual  work  volume  and  workcell  work  hours  in  Figures  41  and  42,  respectively.  For 
an  annued  work  volume  of  28  million  pouches  inspected  over  a  period  of  approximately  280  days,  reducing 
the  unit  workcell  price  from  $56,500  to  $26,000  will  shorten  the  simple  payba’.jc  period  from  3.1  years  to 
approximately  1.5  years.  Some  combinations  of  workcell  costs  and  annual  work  volumes  do  not  allow  the 
payback  to  occm  within  3  years  (upper  right  comer  of  Figme  41).  However,  as  shown  in  the  lower  left 
comer  of  Figure  41,  some  combinations  of  workcell  costs  and  work  voliunes  may  expect  the  payback  within 
2  years. 

As  expected,  one  of  the  most  significant  factors  which  affect  the  retiim  on  investment  in  this  project 
is  the  number  of  hours  the  workceU  is  operated  per  day.  At  $56,500  per  unit  cost,  a  payback  period  of  1.9 
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Figxire  42:  EARW  return  on  investment  output  for  flexible  automation  system 


Figure  43;  EARW  simple  payback  output  controlled  by  working  hours. 

years  can  be  expected  if  the  workceU  is  operated  16  hours  per  day  (Figtire  43).  This  is  mainly  because 
the  number  of  workceUs  required  to  maintain  the  productivity  level  is  reduced.  With  the  16  hour  work 
day  schedule,  if  the  workceUs  cein  be  made  available  at  $26,000  per  unit,  the  simple  payback  peric  l  wUl 
become  less  than  1.5  years. 

6.3  Conclusions 

The  foUowing  conclusions  are  drawn  from  this  study: 

1.  The  munerical  models  developed  in  this  study  can  simiilate,  with  satisfactory  accuracy,  the  MRE 
pouch  inspection  lines  employing  either  human  inspectors  or  automated  workceUs. 

2.  A  conceptual  design  of  a  robot-based  automated  MRE  pouch  inspection  workceU  as  weU  as  a  func¬ 
tional  end-  effector  was  developed  and  evaluated.  The  workceU  was  found  technologicaUy  feasible; 
however,  further  experimental  work  is  needed  to  field  test  the  effectiveness  of  the  end-effector. 
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Figure  44:  EARW  retuiu  on  investment  controlled  by  working  hours. 

3.  The  ranges  of  design  and  operating  parameters  which  would  provide  economic  feasibility  for  auto¬ 
mated  inspection  workcells,  including  flexible  eind  h3ud  automation,  have  been  identifled. 

4.  Using  the  operating  parameters  obtained  from  ein  example  MRE  manufacturer  as  the  baseline  vedues, 
it  is  reasonable  to  expect  that  the  payback  on  the  investment  can  occiu  within  two  years  if  the 
workceUs  operate  10  hours  a  day  cind  their  unit  cost  is  less  than  $50,000. 

This  suggests  that  a  hard  automation  scheme  is  preferable  to  a  flexible  automation  scheme.  Figure 
45  shows  the  einalysis  for  hard  automation  systems  priced  at  $40,000.  If  the  price  is  reduced  as  the 
quantity  increases  (Figure  46),  then  the  return  on  investment  improves  with  payback  in  1.7  years  as 
shown  in  Figure  47. 


106 


Bdsed  on  the  given  velues  : 

Annuil  Ubrk  tbluM  28QGB.09  x  IGGB  uiits/yetf 

Niuto-  of  yORKEI  tveiltbla  :  22.GGB 

NuMber  of  U<(1L  avei labia  :  41.QEB 


Manual 

AutOMtiC 

Task  v«.  Task 

Given  working  hours/day 

19.03 

18.0B  hrs/day 

Hrly  Unit  Uork  Sate 
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24E.857  units/lir 

Daily  Unit  Ibrk  Kata 

4455.009 

2468.572  units/day 

Daily  Uork  Kata 

98.819 

191.211  X  10OD  units/day 

nunber  of  DAVs  required 

28S.E8S 

276.649  days 

Total  Labor  Cost 

X  1000  $ 

ToUl  Fixed  Cost 

1648.0OB  xl8GB$ 

Total  Operating  Cost 

Expected  Payback  Vear  *1 

2.331  years 

Keturn  On  Investeent 

41.626  X 

•1:  ERQI  la  6.0B  x  *2 :  Systen  LlfaTIn  la  IB  years 

Presa  (Enter)  to  continue.. 


Figrire  45:  EARW  output  for  hard  automation  solution. 


Ibnu  of  Changing  Paraieters 
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7 
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hr 

18 

4.  Task  inefficiency. 
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.875 
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1008$ 

38 
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12 
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8.  Task  Cycle  Tim, 
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14 
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hr 

18 

A.  Task  inefficiency. 
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1 

B.  Annual  Ubrk  tbiune. 

10GB  Units 
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General  H 

C.  Kequired  Ubrk  rate. 

IGGB  Units/day 

108 

info. 

D.  Exited  Ketum  On  Investnent 

deciMl 

8 

1 
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18 

Hit  a  No.  to  change  or  (REIUiiN)  to  leave. 


Figure  46:  EARW  peirameters  for  reduced  cost  hard  automation. 
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Based  on  the  gi««n  values  : 

Annual  Usrk  Iblune 
Huadier  of  UOKKER  available  : 
HuMber  of  IKCLL  available  : 


X  1EQ3  units/year 

22.0EB 

dl.6GB 


Hanual  flutoMtlc 

Task  WB.  Task 

Given  yorklng  hours/day 
Hriy  Unit  Ubrk  Rate 
Daily  Unit  Uork  Rate 
Daily  Uork  Rate 
nuidier  of  Dflft  re<iuired 

18.GB 

445.5EB 

44S5.GGB 

98.eiB 

285.685 

18.0B  hrs/day 

246.857  units/hr 
2468.572  units/day 
181.211  X  leee  units/day 
276.649  days 

Total  Labor  Cost 

Total  Fixed  Cost 
Total  Operating  Cost 
Expected  Payba^  Vear  ■! 
Return  On  Investient  *2 

■ 

xl8BB$ 

1238.888  xl8B8$ 

61.588  X  1888  $ 

1.699  years 

58.376  X 

•1:  EXOl  is  B.OBx  *2:  SysteN  LifeTlM  is  IB  years 

Press  <Enter>  to  continue.. 


Figure  47 :  EARW  output  for  reduced  cost  hard  automation. 
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7  Conclusions  and  Recommendations 


The  pvirpose  of  this  short  term  project  STP  li  was  to  determine  the  techniccd  and  economic  feasibility  of 
using  machine  vision  and  robotics  in  packaged  food  manufacturing.  The  results  of  our  reseEirch  suggest 
that  this  approach  is  both  technically  and  economically  feasible;  a  combination  of  focused  illumination  of 
the  packages  with  transport  done  by  fixed  automation  yields  a  system  with  greater  thm  95%  accuracy 
that  can  be  paid  back  in  2  years  with  an  ROI  of  40%. 

7.1  Technical  Outcomes 

Two  simulation  tools  were  developed,  namely  a  numerical  simulation  and  a  graphics  simulation.  For  the 
numerical  simulation,  an  object  oriented  library  was  created.  Such  a  library  allows  the  design  of  numerical 
simulation  for  sever 2il  workcells  by  simply  interconnecting  the  building  blocks  (objects).  Each  piece  was  an 
object  class  written  in  C++.  In  this  way,  the  simulation  of  small  and  large  systems  was  straightforward. 

Information  from  an  existing  MRE  factory  located  in  Texas  was  used  to  create  a  numerical  simulation 
model.  The  simulated  model  was  fine-tuned  until  it  matched  the  average  behavior  in  terms  of  throughput 
and  error  rate.  Then,  the  model  was  extrapolated  to  match  the  average  performance  of  two  other  compa¬ 
nies.  A  second  simulation  based  on  typical  robotic  workcell  parameters  was  implemented.  The  topology 
(i.e.,  configuration)  of  the  robot-based  simulation  was  manipulated  until  its  pe-formance  matched  the 
average  of  the  three  companies. 

It  was  established  that  the  overall  topology  of  the  inspection  plant  needs  to  be  changed  in  order  to  use 
robotic  inspection  workcells.  The  reason  for  this  is  to  increase  the  number  of  paraRel  inspection  lines  to 
compensate  for  (individuEilly)  slower  automatic  inspectors.  If  detection  task  accuracy  is  maintained  (aind 
improved)  by  using  machine  inspection  systems,  then  product  throughput  can  be  mainteiined  even  with 
workcells  that  are  individually  slower. 

Even  though,  at  this  point  of  the  rese^ch,  there  cire  not  sufficient  known  parzimeters  to  biiild  a  complete 
simulation  model,  the  simulation  tool  developed  here  has  proved  to  be  of  great  aid.  Once  a  working. 
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dynamic  prototype  of  the  complete  workcell  is  available,  it  will  be  possible  to  create  an  accurate  simulation 
model  with  data  gathered  from  experimentation.  Such  a  simulation  cam  then  be  extrapolated  to  regions 
(of  the  parzimeter  space)  were  it  is  difficult  or  impossible  to  take  the  read  prototype.  This  close-couple, 
protot5rpe  and  its  simulation,  can  be  used  to  design  an  optimal  robotic  and  machine  vision-based  workcell. 

A  second  simulation  tool  created  was  an  interactive  graphics  simulation.  The  user  was  allowed  to  “fly” 
through  the  graphics  model  to  visualize  all  its  3D  aspects.  A  library  of  graphicadly  modeled  objects  was 
created  so  that  the  overall  workcell  was  e2isily  built.  A  human-based  and  a  robot-based  workcells  were 
simulated. 

By  interacting  with  these  graphics  models  it  was  clear  that  robot-based  workcells  are  Icirger  in  size  and 
require  a  completely  new  layout  distribution.  Some  of  the  conclusions  of  the  numerical  simulation  were 
reproduced  in  the  graphics  simulation.  For  instance,  two  parallel  inspection  lines  were  used  for  the  robotic 
workceU  while  only  one  was  used  for  the  human  workcell.  This  was  done  in  order  to  handle  the  same 
number  of  pouches  per  second. 

The  flexibility  given  by  graphics  cind  numerical  simulation  represents  a  powerful  tool  for  future  imple¬ 
mentation.  The  models  presented  in  this  report  cem  be  considered  as  demonstrative  ones.  However,  the 
same  tools  developed  eind  used  by  this  reseeirch  cein  eilso  be  used  in  the  future  when  the  real  implementation 
will  be  attempted. 

Real  (non-simulated)  models  of  robot  manipulators  were  discussed.  The  two  limit  cases  of  automation 
-flexible  and  fixed—  were  presented.  Out  of  all  possible  implementations  two  particulair  hcirdware  models 
were  discussed.  One  based  on  industrial  robot  arms  (flexible-automation)  and  one  on  customized  designed 
devices  (hzird-automation). 

It  was  established  that  despite  of  the  generality  aind  easy  implementation  of  flexible-automation  it  can 
be  very  expensive.  On  the  other  hamd,  the  flxed-automated  version  presented  a  complex  implementation 
since  designing  zmd  prototyping  must  be  performed  before  any  device  is  integrated  in  a  plant. 

From  this  analysis  it  was  cle2ir  that  a  balance  between  fixed-automation  and  flexible-automation  must 
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be  obtained.  In  this  way  it  wiU  be  possible  to  design  an  optimal  automatic  workcell  teikes  advantage  of 
both  methods. 

Mzdcing  use  of  currently  available  machinery  can  decrease  the  cost  cind  complexity  of  the  workcell 
implementation.  For  instance,  the  problem  of  precise  grasping  (known  orientation  and  position)  can  be 
simplified  if  the  robot  systems  are  tightly  coupled  with  the  automatic  machine  used  to  process  the  pouches 
(currently  used  filler-machine).  In  this  case  the  robot  system  can  make  use  of  whatever  mechanisms  the 
fUler-machine  uses  to  hold  steady  the  empty  pouch,  fill  it  and  seal  it.  This  pEirticular  solution  requires 
further  analysis  of  the  currently  used  machinery.  Such  analysis  is  recommended  to  be  done  in  the  future. 

Finally  the  most  critical  component  of  the  MRE  inspection  workcell  was  analyzed  -the  machine  vision 
system-.  It  was  estabhshed  that  in  order  to  detect  the  broad  spectrmn  of  the  pouch’s  defects,  several 
different  techniques  zire  required. 

A  detection  model  for  defects  that  exposes  the  pouch’s  meted  coating  was  discussed.  Theoretical  analysis 
showed  why  the  model  works  and  prototyping  confirmed  it. 

The  conclusion  of  the  testing  of  the  system  protot3rpe  was  that  the  error-rate  must  decrease  before  it 
is  applicable.  However,  severed  techniques  (based  on  the  theoretical  analysis)  to  prove  the  system  were 
presented. 

There  are  stiU  many  other  vision  techniques  that  have  not  yet  been  explored.  It  may  be  interesting 
to  analyze  and  perhaps  experiment  with  some  of  them.  For  example,  the  use  of  other  interveds  of  the 
electromagnetic  spectrum.  The  electromagnetic  behavior  of  metal  coating  as  opposed  to  plastic  or  pednt 
coating  could  facilitate  the  use  of  other  electromagnetic  field  wave  lengths.  One  can  speculate  and  tcdk 
about  a  micro  wave  based  scanner  (radar  like)  to  located  meted  coating  exposure  (abrasion,  deleimination, 
etc)  or  einomalies  (cut,  holes,  wrinkles,  etc).  Nevertheless,  such  reseeirch  is  beyond  the  scope  of  this 
feeisibility  study. 

Polarized  light  can  be  used  to  increase  the  defect  detection  probabihty.  It  is  known  that  for  every 
matericd  there  exists  cin  angle  Op  called  the  polarization  angle  such  that  all  the  reflected  light  (especially 
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specular-refiected)  at  that  angle  is  linearly  polarize  (in  a  plane  parallel  to  the  reflector  surface).  In  1812 
Sir  David  Brewster  established  that  that: 

Op  =  n^ln-i 

where  ni  is  the  index  of  refraction  of  the  Jiir  zind  n-i  is  the  index  of  refraction  of  the  surface  material.  This 
phenomenon  could,  in  principle,  be  added  to  the  multiple-illumination  and  color-subtractive  technique 
to  remove  in  a  more  efficient  way  the  reflection  due  to  defect-free  surfaces.  This  will  be  plausible  if  the 
refraction  index  of  the  meted  coating  is  different  from  that  of  the  pednt  or  plastic  coating,  so  that,  the  two 
materizds  will  present  a  different  polarization  angle.  In  this  way  it  is  possible,  by  placing  a  lineEir  pol^izer 
in  front  of  the  camera  zind  light  source  (at  the  correct  pol2trization  angle),  to  remove  almost  completely 
one  of  the  material  reflections  (idezdly  the  one  from  the  non-metal  material). 

Even  though  it  may  be  impossible  to  remove  completely  this  component,  the  system  will  decrease  it, 
improving  in  this  way  the  error  detection  rate.  Such  a  system  was  tested  successfully  for  the  purpose 
of  this  reseaurch  in  a  few  pouch  samples,  however,  so  many  new  parameters  were  introduced  that  a  full 
development  of  a  first  optimal  prototype  would  require  several  months  of  ainalysis  and  experimentation. 
The  investigators  consider  this  technique  to  represent  a  good  follow  up  to  the  technique  described  in  section 
five. 

More  technical  data  on  the  MRE  pouch’s  material  must  be  gathered  before  new  techniques  are  at¬ 
tempted.  For  instance,  exact  material  composition  and  its  interaction  with  electromagnetic  radiation.  It  is 
important  for  the  development  of  analytical  models  and  for  efficient  prototyping  to  have  a  full  description 
of  the  material  that  composes  the  pouch.  For  instance,  in  order  to  determine  the  optim2d  color  for  the  filter 
used  by  the  color-subtractive  technique,  it  is  important  to  know  the  correct  experimental  frequency  spec¬ 
trum  of  the  “green”  coating.  In  order  to  determine  the  correct  field  of  illumination  to  be  used  (bright-field 
or  dark-field  model)  combined  with  polarization  light  techniques,  it  is  importmt  to  know  the  refraction 
index  of  the  “green”  coating  and  the  metal  coating.  AU  this  information  cm  be  obtziined  if  the  pouch  is 
analyzed  in  a  matericils  laboratory. 
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New  techniques  must  be  developed  so  that  it  is  possible  to  deal  with  other  types  of  defects.  Tactile 
sensors  could  be  used  to  scan  the  seal’s  surface  looking  for  wrinkles,  material  entrapped  on  the  seal,  defective 
tear  notches,  etc.  A  two-finger  like  robot  end-effector  could  easily  slide  over  the  seal  while  reading  the 
thickness  of  it.  Wrinkles  aind  entrapped  material  are  definitively  notable  disturbances  of  the  seal  thickness. 
The  same  device  can  be  used  to  detect  the  position  and  size  of  the  tear  notch.  Such  a  device  C2in  be 
integrated  together  with  the  trsmsport  system  so  that  no  extra  time  is  added  to  the  inspection  process. 

Once  the  different  components  have  been  optimally  determined  it  will  be  of  great  import£ince  ele¬ 
ment  a  complete  robotic  workceU.  This  will  allow  to  determine  overall  behavior  of  the  system.  Experimen¬ 
tation  on  the  real  prototype  will  determine  the  critical  points  to  be  strengthened.  Experiments  will  also 
determine  the  real  characteristics  of  the  workcell.  Such  characteristics  can  be  used  to  feed  the  niunerical 
and  graphical  simulator.  Simulators  can  be  then  used  again  to  model  the  complete  plant  so  that  a  more 
rezdistic  topology  and  layout  of  a  plant  can  be  determined. 

7.2  Recommendations 

In  summary,  our  theoretical  work,  simulations,  experiments  and  economic  analysis  suggest  that  automated 
inspection  of  MRE  pouches  is  technicedly  and  economically  feasible.  Technical  feasibility  is  based  on 
fundzunental  principles  of  the  interaction  of  timed  radiation  (colored  light)  and  the  surface  under  test. 
Our  detection  results  would  not  be  achievable  by  image  processing  alone,  but  by  paying  attention  to 
physical  chsuacteristics,  one  achieve  both  feiwibility  emd  efficiency.  Coupling  the  image  emalysis  with  fixed 
automation  for  product  transport  will  make  the  system  amenable  to  use  in  a  production  environment. 
Our  recommendations  for  the  next  step  are: 

1.  The  problem  of  MRE  pouch  placement  and  transport  was  relatively  easy  to  solve  in  simulation, 
but  a  prototype  needs  to  be  built  to  conduct  experiments  to  verify  that  the  rates  predicted  by  the 
simulation  are  achievable. 

2.  It  was  established  that  the  detection  system  is  feasible,  but  needs  to  be  improved.  There  are  no  false 
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positive  defects  detected,  but  some  (5%)  of  the  true  defects  went  undetected.  These  defects  were  on 
pouch  locations  of  high  surface  curvature;  the  loss  is  likely  due  to  shading  introduced.  Polarizing  the 
colored  light  is  an  exciting  possibility  for  fine  tuning  the  system  to  avoid  losing  these  kinds  of  defects 
and  should  be  studied  in  the  next  phase. 

3.  Not  only  should  the  accuracy  of  the  system  be  studied,  but  the  inspection  time  can  be  improved. 
The  hardware  used  for  the  timing  in  this  report  is  vintage  1990;  even  in  the  last  4  years  there  have 
been  such  drzunatic  changes  in  the  kinds  of  special  purpose  computational  heurdware  available  that 
the  7  second  figure,  although  a  baseline  is  very  pessimistic  by  todays  standards.  We  recommend  that 
a  prototype  be  constructed  from  those  special  ptirpose  image  processing  boards  available  now  (1994), 
rather  than  the  specifications  used  herein,  zdthough  even  by  the  older  standards  the  underlying 
approach  is  economically  feasible. 
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A  Simulation  Code 


The  following  list  is  the  code  for  the  complete  pre-retort  and  post-retort  models  of  the  Texas  plant  using 

SOL. 

//  Default  *4i4i***4i4i**<|i*«4>**<k*4i****4>««4i*«*i*4>******«**4i4i*4>*>t<4>4i*** 

iinclude  <stdio.h> 
iinclude  <stream.h> 
iinclude  <math.h> 
iinclude  <stdlib.h> 
iinclude  <time.h> 
iinclude  <ctype.h> 

//  STEP  1  <t>it>e**e*****«**»**4i*********«4i***************>t"t>**«**** 

iinclude  "sol.h" 
iinclude  "bclasses.h" 
iinclude  "sim.h" 

mainO 

{ 

//  STEP  2  *4>*4>****«***«***«***«**«4i*4<*******«***4i***«******«* 

Source  source(3) [4] ;  //rate  1  pouch/sec 

Conveyor  convey_l(6) [4] ;  //cap.  6  pouches. 

Inspector  inspector(l) [8] ;  //insp.  time  1  pouch/sec 

Conveyor  convey_2(6) [4] ;  //  cap.  6  pouches 

Transfer  transferCl , . 1) [4] ;  //cap.l  item,  speed.  Isec/item 
Buffer  buffer.AdS)  [4] ;  //cap.  15  pouches. 

Buffer  buffer.BdS)  [4] ;  //cap.  15  pouches. 

Transfer  transf _Ad , . 1) [4] ;  //capacity  1  item,  sped:  3sec/item 

Transfer  transf _Bd ,. 1) [4] ;  //capacity  1  item,  sped:  3sec/item 

Accepted  accepted;  //None. 

Disposed  disposed;  //None. 

//  STEP  3  ************************’¥************************** 
install (  ftsource[0],  10  ); install (  ftsource[l],  11  ); 

instalK  ftsource[2],  12  );install(  ftsource[3]  ,  13  ); 

instalK  ftconvey.l [0] ,  40  );install(  &convey_l[l] ,  41  ); 

instalK  ftconvey_l[2]  ,  42  );  instalK  &convey_l  [3]  ,  43  ); 

instalK  ftinspector [0]  ,  50  );instalK  Ainspector[l]  ,  51  ); 
instalK  ftinspector  [2]  ,  52  );  instalK  Ainspector[3]  ,  53  ); 

instalK  ftinspector  [4]  ,  54  );  instalK  Ainspector[5]  ,  55  ); 

instalK  ftinspector  [6]  ,  56  );  instalK  &inspector[7]  ,  57  ); 

instalK  ftconvey_2[0]  ,  401);instalK  &convey_2[l]  ,  411); 

instalK  ftconvey_2[2]  ,  421);instalK  &convey_2 [3]  ,  431); 

instalK  fttransferCO]  ,  30  );instalK  AtransferCl]  ,  31  ); 

instalK  Atransfer[2]  ,  32  );  instalK  Atransf  er  [3]  ,  33  ); 

instalK  Abuffer_A[0]  ,  20  );instalK  Abuffer_A[l]  ,  21  ); 

instalK  tbuffer_A[2]  ,  22  );  instalK  Abuffer_A[3]  ,  23  ); 

instalK  ftbuffer.B[0]  ,  24  );instalK  Abuffer.BCl]  ,  25  ); 

instalK  Abuffer_B[2]  ,  26  );instalK  &buffer_BC3]  ,  27  ); 

instalK  Atransf_A[0]  ,  301);instalK  fttransf _A [1]  ,  311); 

instalK  Atransf _A[2]  ,  321);instalK  fttransf _A[3]  ,  331); 
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instalK  jttransl_B[0]  ,  302)  ;install(  fttransf.BCl]  ,  312); 

instalK  fttransf _B[2] ,  322) ;install(  fttransf _B[3] ,  332); 

instalK  ^accepted,  60  );  instalK  ^disposed,  70  ); 

//  STEP  4  **««»************<ti*««******«*««*******««*******««it<>t< 

//  LINE  1  //  //  LINE  2  // 

connectdO,  40,  UNDEFINED);  connact(ll,  41  .UNDEFINED) ; 
connect(40,  50,  UNDEFINED);  connect(41.  52, UNDEFINED) ; 
connect(40,  51.  UNDEFINED);  connect(41,  53, UNDEFINED ) ; 
connect(50,  401 .UNDEFINED) ;  connect (52. 411, UNDEFINED) ; 
connect(51,  401 .UNDEFINED) ;  connect (53, 411. UNDEFINED) ; 
connect (40 1,30,  UNDEFINED);  connect (411, 31. UNDEFINED) ; 
connect(30,  20,  GOOD);  connect(31,  21. GOOD); 

connect (30,  24,  BAD);  connect (31,  25, BAD); 

connect(20,  301 .UNDEFINED) ;  connect(21 ,311. UNDEFINED) ; 
connect(24,  302, UNDEFINED) ;  connect (25 ,312, UNDEFINED) ; 
connect (301, 60.  UNDEFINED);  connect (311 .60. UNDEFINED) ; 
connect (302, 70,  UNDEFINED);  connect (312, 70, UNDEFINED) ; 

//  LINE  3  //  //  LINE  4  // 

connect(12,  42,  UNDEFINED);  connect(13,  43, UNDEFINED) ; 
connect(42,  54,  UNDEFINED);  connect(43,  56, UNDEFINED) ; 
connect (42,  55,  UNDEFINED);  connect (43,  57, UNDEFINED) ; 
connect(54,  421 .UNDEFINED) ;  connect(56,431,UNDEFINED) ; 
connect(55,  421 .UNDEFINED) ;  connect(57,431,UNDEFINED) ; 
connect (42 1,32,  UNDEFINED);  connect(431 ,33.UNDEFINED) ; 
connect(32,  22,  GOOD);  connect(33,  23, GOOD); 

connect(32,  26,  BAD);  connect(33,  27, BAD); 

connect (22. 321,  UNDEFINED);  connect (23, 331, UNDEFINED ) ; 
connect (26, 322,  UNDEFINED);  connect(27, 332, UNDEFINED) ; 
connect (321, 60,  UNDEFINED):  connect(331 ,60, UNDEFINED) ; 
connect (322, 70.  UNDEFINED):  connect(332 ,70, UNDEFINED) ; 

//  STEP  5  and  6  *****m*********************m***************** 
defects (2) ; 

TFR[0]  =  0.1; 

TFR[1]  =0.1; 

TFR[2]  =0.1; 

simulate(  3600.0,  .2,  FALSE  ); 

> 
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The  following  is  a  selection  of  the  SOL  software. 

/ /^nnmm****************************************!******************/ / 


//♦ 

*11 

//* 

RUTGERS  UNIVERSITY 

*11 

//* 

MAN-MACHINE  INTERFACE  LABORATORY 

*11 

n* 

*11 

n* 

STPll 

*11 

//* 

SOL  LIBRARY 

*11 

//♦ 

Basic  Classes  Definition 

*11 

//* 

*11 

II*  bclasses. 

•h  Spring  1992 

*11 

/ /**ini***i********************************m************<in>nnf******0‘/ / 

. . . // 

ITEM  — // 

- // 

class  Item 
-[ 

protected; 
long  int  ID; 
int  state; 

double  delect [  DEFECTS _PER_ ITEM  ];  //The  actual  quality, 
double  estimate C  DEFECTS.PER.ITEM  ];  //Given  by  an  inspector. 

public: 

ItemO 

{ 

state  =  UNDEFINED; 

} 

void  set_state(int  st)-(  state  =  st;  } 
void  set_ID(int  number) {  ID  =  number;  }; 
void  set.defects  (double  ’•■del _prob)  ; 
int  show_state(){return(state) ;} 

int  show_ID(){  return ( ID ) ;  }; 

double  ♦show.delectsOf  return(def ect)  ;  }; 
double  show_defect(int  j){return(defect[j] ) ;> 
void  genere.defectsO ; 

}; 

- - - — . _// 

UNIT  CLASS  — // 

- - - - // 

class  Unit 

{ 

protected: 
int  ID; 
int  state; 
int  type.unit ; 
long  int  item.input ; 
long  int  item.output ; 


//— 

//— 

//— 


//— 

II— 

II— 
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Item  item; 


public ; 
Unit ( ) 
{ 


ID  =  0; 
state  =:  IDLE; 

item.input  =  item.output  =  0; 

} 

virtual  int  set_ID(int  id  ){  ID  =  id;  return(  ID  );} 
int  show_ID()-C  return (  ID  );} 

int  set_state(  int  st  ){  state  =  st;  return(  state  );} 
int  show_state(){  return(  state  );  } 
int  show_type()-(  return(  type.unit  );} 

virtual  long  int  show. index (){  return(item_input  -  item.output) ;} 

long  int  show_itmout(){  return(  item.output  );} 

long  int  show_itminp(){  roturn(  item.input  );} 

virtual  int  show.item.state(){  return(  item.show.stateO) ;} 


}; 


// . . . . // 

//—  SOURCE  CLASS  — // 

// . - . - . // 


class  Source  :  public  Unit 

{ 

protected: 
double  issue. last; 
double  issue.per; 
double  issue. delta; 

public : 

Source  0 

type.unit  =  SOURCE.TYPE; 

issue. last  =  0; 

issue.per  =  1/SOURCE.PERIOD ; 

} 

Source (double  rate) 

-C 

type.unit  =  SOURCE.TYPE; 
issue. last  =  0; 
issue.per  =  1.0/rate; 

> 

int  set_ID(int  id  ) 

< 

ID  =  id; 

cout  <<  "Source  installed.  \tID  =  "<<  ID  <<  "\n"; 
return (ID) ; 

} 

long  int  show.index(){  return(item.output) ;}; 

put.itemdtem  ♦out. item) ; 

double  show.per()-(return(issue.per) ;} 

}; 
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// .  II 

1 1—  BUFFER  CLASS  — // 

// - - n 


class  Buffer  :  public  Unit 

{ 

protected : 
int  in_stack_ptr ; 
int  out_stack_ptr ; 
int  capacity; 

Item  stack_itemCMAX_BUFFER_CAPACITY] ; 

publj.c : 

Buffer 0 

{ 

type.unit  =  BUFFER.TYPE; 
state  =  EMPTY ; 
in_stack_ptr  =  0; 
out_stack_ptr  *  0; 
capacity  =  BUFFER_CAPACITY-1 ; 

} 

Buffer(int  cap) 

{ 

type.unit  =  BUFFER.TYPE; 
state  =  EMPTY; 
in.stack.ptr  *  0; 
out.stack.ptr  *  0; 
capacity  =  cap-1; 

} 

int  set_ID(int  id  ) 

< 

ID  =  id; 

cout  <<  "Buffer  installed.  \tID  =  "<<  ID  <<  "\n"; 
return (ID) ; 

} 

int  show_in_ptr(){  return(in_stack_ptr) ;} 
int  show_out_ptr(){  return(out_stack_ptr) ;} 
int  get_item(  Item  in_item  ) ; 
int  put_item(  Item  *out_item) ; 
int  show_item_state() ; 

}; 


//—  TRANSFER  CLASS  — // 

// . - - - H 


class  Transfer  :  public  Unit 

{ 

protected: 

double  last.trzmsfer; 

double  delta.time; 

double  current_transfer_time; 

public: 

Transfer () 
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{ 

type.unit  =  TRANSF.TYPE; 
state  =  EMPTY; 
last.transfer  =  0.0; 

curreiit_transfer_time  =  TRANSFER_TIME;  //4.0  *  RAND  +  2.0; 

}; 

Transfer (int  cap,  double  trans.time) 

{ 

type.unit  =  TRANSF.TYPE; 
state  *  EMPTY; 
last.transfer  -  0.0; 
current.transfer.time  =  trans.time; 

}; 

int  set_ID(int  id  ) 

■C 

ID  =  id; 

cout  <<  "Transfer  installed.  \tID  =  "<<  ID  <<  "\n"; 
return (ID) ; 

} 

int  get.itemC  Item  in.item  ) ; 
int  put_item(  Item  *out_item) ; 


// . . . // 

//-—  CONVEYOR  CLASS  — // 

// . - . // 


class  Conveyor  ;  public  Buffer 

{ 

public: 

Conveyor 0 

capacity  =  CONVEYOR.CAPACITY-l ; 
type.unit  =  CONVEY.TYPE; 
state  =  EMPTY; 

} 

Conveyor (int  cap) 

{ 

capacity  =  cap-1; 
type.unit  =  CONVEY.TYPE; 
state  =  EMPTY; 

> 

int  set. ID (int  id  ) 

< 

ID  =  id; 

cout  <<  "Conveyor  installed.  \tID  =  "<<  ID  <<  "\n"; 
return (ID) ; 

} 

>; 


// . — . - . - - - - - // 

//— -  INSPECTOR  CLASS  — // 

//- . — — - . . . // 


class  Inspector  :  public  Unit 
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public : 


Inspector () 

type.unit  »  INSPEC.TYPE; 
state  =  EMPTY; 

} 

Inspector(int  insp.time) 

{ 

type.unit  =  INSPEC.TYPE; 
state  =  EMPTY ; 

} 

int  set.ID(int  id  ) 

ID  =  id; 

cout  <<  "Inspector  installed.  \tID  =»  "<<  ID  <<  "\n"; 
return (ID) ; 

} 

int  get.item(  Item  in.item  ) ; 
int  put. item (  Item  *out_item) ; 
int  inspection 0 ; 

}; 


// . - . // 

//—  ACCEPTED  CLASS  — // 

// . // 


class  Accepted  :  public  Unit 

■t 

public: 

AcceptedO 

■c 

type.iulit  =  ACCEPT.TYPE; 
state  =  READY; 

> 

int  set_ID(int  id  ) 

ID  =  id; 

cout  <<  "Accepted  installed.  \tID  =  "<<  ID  <<  "\n"; 
return (ID) ; 

} 

int  get.item(  Item  in.item  ) ; 

}; 


// . . . . . // 

//—  DISPOSED  CLASS  — // 

//— - . . . - . - - - - - // 


class  Disposed  :  public  Unit 

■C 

public; 

DisposedO 

{ 
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type.unit  *  DISPOS.TYPE; 
state  ^  READY; 

} 

int  set_ID(int  id  ) 

ID  =  id; 

cout  <<  "Disposed  installed.  \tID  =  "<<  ID  <<  "\n"; 
return (ID) ; 

> 

int  get_item(  Item  in.item  ) ; 

long  int  show.indexO-C  return ( item.input)  ;>; 


//- - - - - - - - - // 

//—  PROCESS  CLASS  — // 

//-- . - . . .  -// 


class  Process  :  public  Unit 

{ 

public: 

Process  0 

i 

type.unit  =  PROCES.TYPE; 
state  =  READY; 

} 

int  set. ID (int  id  ) 

< 

ID  =  id; 

cout  «  "Process  installed.  \tID  =  "<<  ID  «  "\n"; 
return (ID) ; 

} 

int  get_item(  Item  in.item  ) ; 
int  put_item(  Item  •out.item  ) ; 
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